Gmail IMAP vs. Previous POP3 Users

Google Mail now supports IMAP, but what if you’ve been using POP3 all along and have a gajillion messages on the server, all marked unread and waiting in your inbox? How can I tell Apple Mail not to download the [Gmail]/All Mail IMAP folder without an ugly hack? [Update, the hack just causes Mail to crash a lot.]