I've had a few people wonder about this.
If you are doing the exercise, if you are writing and keeping your pen moving forward the entire time, then you are doing it right.
If you stray from the start line it doesn't matter. I always say that where you start isn't really the point, that's just to get you started. What emerges is what needs to emerge. If you start with a line about your protagonist's fear and 1/2 way through you realize you are writing about her father's backstory, keep writing! It's all going to be in there somewhere. It's all you discovering your story. Keep the water moving through the pipes. This is where inspiration and surprise lurk.
Just to show you what I allow in my own writing practice, here's an example of an exercise gone wild. As you can see, I ask myself a lot of questions while I write. I do this all the time. One, it's a great segue into the next thought. Two, it's simply a way to discover the story. And you don't have to know all the answers right away. I also leave blank spaces when I can't think of something, or don't know a certain fact, simply to keep moving. Don't allow yourself to get stuck!
This is a story about adventure. This is a story about survival. This is a story about love. This is a story about taking a risk, about stepping up to the plate and I keep worrying about contradicting something I've already written in the past so I think it's blocking me a little from writing anything new... This is a story about a little Howler Monkey whose brother is taken by Harpy Eagles and he must go to the World of Waiting Spirits to retrieve him and I seem to be missing great chunks of story because I've got 10 pages until his brother disappears and a few more until Howler reaches Stilt and the Cantina but what else happens along the way and why am I always writing about the jungle and I must learn some more animals and I know another animal appears but I'm not sure why I mean usually they all want something from Howler and are there giraffes in South America? No, silly, only in Africa... so Howler is led by the bird to the Cantina and does Stilt accompany him and I think he must make friends with Tiger Cat who should be a female character I need more female characters...
See what I mean? It's a mental purging, a puking on the page, if you will. You have to dig later for the diamonds. (I think I'm mixing my metaphors here. Unless you are prone to throwing up diamonds. In which case, I've got some ipecac for you.)