If you've grown up drinking cow's milk, goat milk can be an acquired taste. Goat milk is stronger in flavor (I've heard it described as "gamey" or "strongly goaty") and has a different aftertaste than cow's milk.
As for me, I've always liked goat milk, and I can't resist picking up a quart when I find it (which isn't very often, BTW - goat milk is still considered a "specialty item" and a lot of supermarkets don't regularly carry it.) For that matter, I like goat milk yogurt and goat cheese, too. Hell, if I thought I could get away with it, I would keep a nanny goat here in my quiet suburban neighborhood and be all Goat Milk Dairyman with her.
Anyway. Different brands of goat milk seem to have different goaty intensities. I recently picked up a quart of Meyenberg Low Fat Goat Milk and really enjoyed it. But I have to say it was much stronger in flavor than some of the other goat milks I've had. It had a very distinctive "barnyard taste," which is something that some people find objectionable but I find interesting.
I would certainly buy it again, especially if I also have Cap'n Crunch in the house. Because that's something else about goat milk: it is awesome on Cap'n Crunch.