The Best Way to Archive E-Mail

I’m a heavy user of e-mail — between 300-400 e-mails a day. There are some definite strengths and weaknesses to the different e-mail platforms.

My primary platform is Microsoft Outlook, and there are some key strengths there. First, it’s the best way to connect to a Microsoft Exchange server. Exchange integrated with BlackBerry is the best mobile e-mail solution on the planet, bar none. (If anybody disagrees, let me know and I’ll write a blog post to foster that debate!) One of the other big strengths with Outlook is that you can use the natural motion of drag-and-drop to attach files to messages, or save messages inside tasks, etc.

My secondary platform for almost all of my other accounts is Gmail, or more accurately, Google Apps (which lets you use Gmail for your own domain name, like aaronklein.com). The Gmail platform has some great strengths too: low costs, incredible speed, amazingly good search (it applies Google’s search technology to your e-mail), and you don’t have to worry about storage space or backup — it’s all in the cloud.

Of course, each platform tries to compensate for the other’s strengths. Google is increasing integration with BlackBerry, but it’s still not nearly as good. Outlook 2007 is now integrated with Windows Desktop Search which is exponentially faster than before, but it doesn’t scale…Outlook still slows way down when you have more than about 2GB of data in a file.

To solve that problem, Outlook has an AutoArchive feature that will transfer older mail to a different Outlook data file. The problem is, that Outlook file will eventually get quite large too. If you leave it open, it slows Outlook down. If you close it, it won’t be indexed by Windows Desktop Search, so it’s practically useless.

What I realized is that I can get the best of both worlds by transferring mail from the Outlook world to the Gmail world as a permanent archive: it lets me shrink my Outlook data files to a manageable size, and keep my archived mail very searchable, accessible and stored in the cloud.

As a sidebar: I don’t keep all of my e-mail messages. There are several organizations I’m involved in where e-mail retention policies are established that determine when messages get deleted. But I end up archiving much of my personal and work related e-mail, and some categories of messages date back to 2002.

Here are the steps I took:

1. I downloaded Google E-Mail Uploader. This only works with Google Apps accounts that run on your own domain name, and won’t work with a Gmail.com account (hopefully they’ll remedy that soon).

2. I created a new Outlook data file. To do that, click on Tools / Account Settings, go to the Data Files tab and click Add. Make sure to name this file something like “Mail to Archive” so it doesn’t get too confusing!

3. I moved the messages that you want to archive from your main folders to that new data file. For safety, I selected the e-mail I wanted to archive, and held down the Ctrl key while dragging it into the corresponding folder in the new data file so that it would copy the messages.

(After verifying that they copied successfully, I then went back and deleted those messages in the primary folder…and then I emptied the deleted items. But I really didn’t want to lose those messages!)

So at this point, you should have the mail you want to archive in the new data file, and all of the mail you want to keep in Outlook in your main data file. I chose November 2008 as my cutoff, and figure that by the next time I need to archive, I’ll know whether to keep 12 or 18 months in Outlook.

4. I launched Google E-Mail Uploader. After closing Outlook, I could select the archive folders I wanted to upload, and it started. The tool preserves your folders by creating the corresponding labels in Gmail. The entire process took 22 hours for about 3.5GB of e-mail messages, but I could easily pause the process, exit the uploader tool, and restart the process later without missing a beat.

Voila! Years of e-mail stored safely online in the Gmail platform. I can easily search it and find old messages, and forward them to my current account if needed.

Have you tackled this problem before? Any twists on this idea that could make it easier for people?