I've been a cat person my whole life (although I haven't had one for several years due to frequent moves and other issues). I can tell you that milk is actually bad for cats; they can't digest it. The might love to drink it, but it often makes them vomit or gives them pain and diarrhea.

So along with not feeding the cat for a variety of other possible reasons, this is a great reason not to give it any milk.