You'll need to make sure you have the necessary codec(s) installed to replay mpeg4 video/audio on windoze media player.
A quick search on the ms site gives this:
http://www.microsoft.com/windows/windowsmedia/forpros/format/codecdownload.aspx
It claims to work for series 9 & 10 players (I've not tried it).
You'll also start to come across m4a extensioned audio files (that's the audio bit of mpeg4). Typically purchased from iTunes and other sites.