Finding the Parenting Sweet Spot

Scenario #1: Father wants to teach child how to swim. Child is terribly afraid of the water. Father doesn't want to throw the child into deep water (as his parents did to him) because he feels this is (a) somewhat cruel and more importantly (b) counter-productive, as the experience will probably traumatize said child and further delay his aquatic career.

Scenario #2: Parents want to go out, leaving 13 year old to babysit his siblings for the first time, including taking them out for pizza, and getting them into bed without fighting with them. Parents are anxious about giving said child such a large level of responsibility. Should they do it?

I've encountered a number of articles about parenting techniques of late, including a fascinating New Yorker piece (a must read!) contrasting the responsibility and maturity of the Matsigenka, a tribe who live in the Peruvian Amazon, and the typical American child who, the author correctly notes, "may represent the most indulged young people in the history of the world." (I would take out the word "may".) It seems clear that the more we give our children, the less they are willing to work, earn and appreciate what they have. And yet, as parents, we seem to want to give them more, when we know that this is probably the very worst thing that we could do for them.
A similar recent NY Times article on the same subject described the optimal parenting technique:

Decades of studies, many of them by Diana Baumrind, a clinical and developmental psychologist at the University of California, Berkeley, have found that the optimal parent is one who is involved and responsive, who sets high expectations but respects her child’s autonomy. These “authoritative parents” appear to hit the sweet spot of parental involvement and generally raise children who do better academically, psychologically and socially than children whose parents are either permissive and less involved, or controlling and more involved.

So, in other words, we're supposed to set expectations, and then let our children go fulfill them, without intervening (and helping them). In essence, we're supposed to challenge our children, and let them struggle with the challenges. Sometimes our children will succeed, sometimes they'll fail, and the process will probably always involve a level of struggle and difficulty. But they need the opportunity to strive in order to grow to become independent people in their own right.
Reading these articles made me think of a passuk in Parshat Eikev and a recent section from the Gemara in Brachot. Describing the trials and tribulations that the Jewish nation faced throughout their forty years in the desert, Moshe uses an interesting, somewhat puzzling analogy:

וְיָדַעְתָּ, עִם-לְבָבֶךָ:  כִּי, כַּאֲשֶׁר יְיַסֵּר אִישׁ אֶת-בְּנוֹ, ה' אֱלֹקיךָ, מְיַסְּרֶךָּ.
And you know in your heart, that just as a man afflicts his son, so too does Hashem your God afflict you.

Actually, the word ייסר - which I translated as "afflict" - is a rather harsh translation. Many of the commentators see the word as being based on the word מוסר - "rebuke", meaning that God "rebuked" the nation of Israel like a father rebukes a child. But the root also emanates from the word ייסורין - or afflictions, which makes us wonder: do parents really afflict their children? Are we supposed to make our children suffer?
I doubt that the New York Times would think so. But in essence, isn't that what parenting is about? What child really wants to toil over difficult math problems? Swimming in the deep end is indeed frightening to a child who's scared of the water. At least at first, learning to swim is a form of suffering. So, should we force the child to suffer, or give in to their desire to abandon the math homework and not learn how to swim?
Obviously, I don't think so. But that doesn't make it any easier.