Update: Mail.app is crap! its buggy and simply doesn’t work well with Gmail IMAP at all, I’ve switched to Postbox which is lightening fast (imported my 5000+ emails in seconds) and has way cooler features than Mail like threaded messages etc.
Seems like loads of people are having the same issue I encountered thinking I’d be clever and use Mail.app rather than Gmails web interface. The main issue is that its SUPER SLOW, and seems to be downloading messages for days on end.
I found a combination of things seems to have sped it up and made it behave more as I would have expected.
Firstly follow the instructions laid out here:
http://db.tidbits.com/article/10253
Secondly disable “Use IDLE command if the server supports it”
http://getsatisfaction.com/apple/topics/mail_app_slow_delay_imap_synchronization
Things should start to behave. All seems to be ok now that my 5000+ emails have all been synced!