With growing interest and amazement, I read the back-and-forth argument between two long-time, highly respected Mac nerds yesterday on the subject of Mark Pilgrim’s decision to abandon Mac OS X for Ubuntu Linux. John Gruber is simply one of the best Mac writers there is, and regardless of what he has to say on a particular subject, you have to admire the elegance, precision, and logic of his writing. So when Gruber raised questions about the wisdom of Pilgrim’s move in a recent blog post, his large readership weighed in, and Pilgrim responded, you can be sure that a great many Mac users like me paid attention.

As usual, I agreed with nearly everything Gruber had to say, and the couple of niggles I have are not worth mentioning here since they would distract from the purpose of this article. And what is that purpose, you are wondering? Before I get to that, let me briefly summarize (if I dare) the exchange so far between Gruber and Pilgrim.

1. Pilgrim has become fed up with Apple’s “closed”-edness. After 22 years as a sophisticated, high-end user, he’s decided Apple’s “closed” ecosystem of software and hardware is too closed for him. His primary concern is that the integrity of the data he stores in that ecosystem is at risk, because Apple doesn’t always document its data formats and doesn’t respect for long the proprietary formats it develops for storage. Pilgrim feels jerked around from one closed format to another and is tired of the data conversions and consequent data loss they inevitably entail.
2. Gruber is surprised and a bit incredulous that Pilgrim would have suddenly been bitten by this bug. He agrees that closed formats aren’t good for long-term archival purposes, but questions whether losing his iTunes metadata and other format problems is worth chucking his expertise with the Mac operating system for something completely different. He points out that a good backup strategy is part of the solution to preserving precious content. He also devotes a large part of his response to criticizing the Mac blog writers who had knee-jerk reactions against Pilgrim’s decision, and who cited old “Mac is better than Windows because…” arguments without realizing the advances Windows has made since Windows XP (or 95, or whatever). Gruber argues against black-and-white thinking in general and for the very reasonable position of respecting other people’s choices even if you don’t agree with them.
3. Pilgrim replies that Gruber missed his point and reemphasizes that his feeling “closed in” by proprietary formats has been coming on for a long time. Apple’s decision to abandon the widely used and understood mbox format for Mail was just the last straw. He feels betrayed that Apple switched formats in Tiger without informing its users, without providing them a way to back out, and without documenting the new format.

So why do I want to wander into this disagreement between two Macintosh heavyweights I don’t know, but greatly admire and respect? As I read their separate articles, I saw something with my Martian eyes that may not be clear to them. What I saw wasn’t an OS switch story, but rather a love story.

I’m coming to believe that the human brain just isn’t very reliable. Long ago I concluded that humans would never understand their own behavior, simply because they’re not capable of analyzing behavior without affecting it. The mind is too complex, there are too many variables that define behavior, and how can you stand outside human-ness and study it without reflecting your own beliefs and preconceptions? It just can’t be done, which is why we’ve made so little progress in psychotherapy and instead are becoming more and more dependent on the “objective” injection of drugs (which we also don’t fully understand).

So the prospect of someone as intelligent and knowledgeable (those are different things) as Mark Pilgrim abandoning an OS as highly evolved as Mac OS X over a file format issue is incredible. This is clearly an emotional response, and that’s the only way I can understand it. As a heavily invested Apple user myself, I have become incensed at the way Apple often treats its customer base. I’ve only been a Mac user for 10 years–less than half of Pilgrim’s 22–but I definitely feel Apple “owes me” something for my “loyalty”, especially after not abandoning the platform in the sad years of the late 1990’s when Apple lost its way. Like Pilgrim–and Gruber–I have many criticisms of the Apple platform and software, and if that Automator action interrupts my work one more time I’m gonna scream! (Why can’t it work in the background when it doesn’t require any input from me?) I have tried–and abandoned–and tried–and abandoned–using a local iDisk with .Mac so many times it’s not at all funny. Each time Apple says they’re improving webdav for .Mac, my hopes go up, and I try again. I’m currently trying again, in fact… we’ll see how long it lasts.

One of the advantages of being a Martian, though, is that I do have the ability to stand outside myself and see when I’m being silly. (It’s the antennae.) So I would recognize when my “fed-up”-ness was wresting control of my good judgment and rein it in. In this case, I think Pilgrim has failed to recognize that his beef isn’t with Mac OS X or the physical entity he calls his Mac, but rather it’s with the faceless corporate entity called Apple, run by its arrogant and righteous managers and programmers.

As Gruber points out, the question isn’t whether Apple is open or not, but whether they’re open enough. One of the things I value about Apple and its products is the new ideas they bring to computing and their willingness to take risks with new approaches. I’m a “love new stuff” kind of guy, so I welcome anything new with open arms. This can be a problem when the new thing turns out to be a skunk in disguise, but with Apple that’s pretty rare. As a New-loving guy, though, I realize that nothing lasts forever. New things always displace old things, and they’re only new for a short time.

As I reminded a web developer colleague of mine recently, when you’re in the business of building websites, you simply have to accept the fact that whatever you think you know today is not going to be sufficient 2 years from now. API’s change, languages change, programming techniques change, graphics technology changes, browser technology changes, hardware capabilities change… you name it! It’s all malleable, and you have to be able to roll with it.

This constant “newness” in computing means that if you want to play and create in the digital world, you have to be prepared to convert from one format to another many times over in the course of your lifetime, or risk leaving valuable creations behind, locked in some old file format (or hardware format) nothing can read anymore. (The alternative, of course, is to stick with easels and canvas and pencil and paper.) I can easily identify with Pilgrim’s concern here… all creative individuals can. What we make we want to keep (the good stuff, anyway), and we want to be able to enjoy it again 5 years from now, or whenever the mood strikes. (I better do something about that large reel of magnetic tape with the original copies of my 1978 recordings on it before it’s too late!) Like Pilgrim, this is a constant worry. I try not to go into any new technology or tool without understanding how I’ll get my content out again. If I find I can’t, I abandon the technology before I’ve invested more of myself than can be easily migrated manually.

A couple of recent examples from my world… Bloglines is a great RSS service, and after having used NetNewsWire for about a year, I switched because Bloglines had this very cool “clip” function. In Bloglines, you can easily “clip” an article and assign it to a folder. Sounds like a great way to archive content you’re interested in, no? No. It turns out Bloglines provides no way either to present that content except through their administrative interface, and no way to export the metadata the clips represent. So, bye bye Bloglines.

Del.icio.us is another terrific service for Web 2.0, and I still use it. But when a series of service interruptions meant I had no access to my bookmarks a few days last fall, I panicked. I used a free plugin to Wordpress called Mysqlicious to import all of my Del.icio.us content and metadata into a MySQL database, and I now keep a mirror of all my Del.icio.us bookmarks there.

In fact, these two conversions are what led to the evolution of Musings from Mars. The “library” of News, Resources, and Software I keep here is nothing more than my bookmarks, all grown up with a great deal more content and metadata than I could store in Del.icio.us or Bloglines. I’m completely at ease now, because relational databases and SQL are standard, well understood data stores and methods of retrieval. The content itself is simply Unicode text, in some cases tagged with HTML. As a web guy, I’m very comfortable with HTML as an archival storage method, and with standard graphics formats like GIF, JPEG, PNG, and TIFF to store the images.

So, what about some of the conversions Pilgrim has had trouble with? Mail, for example? I’ve had lots of fun with mail formats over the years, but it’s possible that the solutions that work for me just wouldn’t do it for him. First, regarding Apple’s new “closed” format that replaces mbox. Is this such a tragedy? I’m of the mind that the engineers at Apple are a lot smarter than me about this kind of thing, and if they felt the need to change mbox in order to optimize Spotlight, I say, go for it! Being able to search across all of my past email is the most important thing anyway, isn’t it? What’s the good of having your mail in an “open” format, if it’s not easy to search? Pilgrim’s beef seems very strange to me, especially after I did a couple of quick experiments this morning with the .emlx format. (Actually, it looks to me like Apple still uses mbox for the mailboxes themselves, and emlx for the individual mail items. In any case, .emlx refers to the mail items, not to the mailboxes.)

First, I tried a new (to me) piece of shareware called File Juicer just to see what would happen when I fed it a folder-full of .emlx files. I was pleasantly surprised to find that everything converted very neatly to .txt, .html, .gif, .jpg, etc files. File Juicer put each file type in its own folder at the end. Lo and behold, all those ads I’d trashed still had their HTML files gloriously preserved! All of the attachments were neatly dumped out for me. Now, this isn’t an email archive, but if it’s the content you want to get at, it’s certainly an easy way to do that.

Second, I used a tried-and-true piece of shareware called Emailchemy, which I had previously used when archiving my Exchange mail to Apple Mail last year. Emailchemy makes it wonderfully easy to convert from just about any email format to another. You just point it at your Mail folder, and Emailchemy will preserve the directory structure, setting up mbox files, Eudora files, Thunderbird files, and more, which you can then import into another mail program. If you want to use a Mail client interface for accessing your archived mail, this is a very easy way to do that. It didn’t take long to import my Apple Mail mail into the terrific Opera mail client this way.

Third, I went ahead and tried a piece of freeware I’d downloaded earlier this year called MHonArc, which is specifically designed to convert from email formats to HTML as an archival utility. Now, this is thinking outside the box, folks! MHonArc is a command-line utility, but there’s a Mac OS X interface (also free) that makes it easy to use without having to learn the command syntax. You just point the software at your mail files, and it converts them to linked HTML. After I combined my “sent” and “inbox” folder in the same archive, I really saw the wisdom of this approach. Since the software preserves threads, now I could easily find my replies and my receipts in the same thread! And honestly, I think data in HTML is pretty darn safe!

A last option I didn’t bother to try, but which I find also very compelling is a tool like MailSteward, which converts your email to a relational database. Now honestly, Mark, doesn’t this sound better than switching to Linux? For $50, you can sock your email in a safe database from which you can output text files, SQL files, mbox files (yes!), and print (PDF) files, and which you can search much more flexibly than any Mail client can.

See, Mark… it’s not Mail, and it’s not iTunes, and it’s not AppleWriter, or whatever. Slowly, over many years of frustrations, you’ve developed a negative attitude toward Apple, and the bough has now broken. Apple is a lucky company in that they instill intense loyalty, verging on worship, in their users. But like love, loyalty has a dark side. Like someone we love who has betrayed us, Apple has a way of pissing off its loyal customers through neglect and indifference. Everyone who has used the Apple support forums has found them useful, but also quite cold. I have never ever seen an Apple employee step in to one of them to help people out. In fact, they seem to deliberately avoid doing so. Not good PR, in my view.

After having spent $7,000 in one year at my local Apple store, I became livid at a manager there who refused to give me my Federal employee discount, simply because I’d forgot to bring it to the store with me. I said, “Just look me up in your database.” He said, “We don’t have access to that information.” “What!? How can you provide top-notch customer service when you have no way of getting to know your customers?” He just shrugged his shoulders and acted like he didn’t know, didn’t care.

If Apple’s customers sometimes feel like they have a personal relationship with the company, then it can be a particularly bruising relationship for guys like Pilgrim who are digitally gifted and technologically savvy. It would be like being married to a spouse who you admire because they’re a little bit smarter than you. The spouse does many wonderful things and makes many fine decisions. But lots of decisions get made without consulting you. You argue about this, and she promises to get your opinion the next time before going off in a different direction. But then she doesn’t. Never does, in fact. Even when you think she’s brilliant, you bristle at her superior attitude. Finally, it’s too much for your ego, and off you go.

For “the rest of us,” Apple is simply brilliant so much of the time that we just forgive the few blunders. I personally stand in awe of so much about Mac OS X that I could give a rat’s ass about the mailbox format it uses. My issue is, “Make Mail play nice with Exchange 2000!” I’m delighted that Apple uses XML as a core data format, including the format for my iTunes metadata. I’m in love with QuickTime because it’s completely interoperable with open video and audio standards, and it’s extremely easy to convert from one format to another. Unlike Microsoft, Apple hasn’t tried to develop its own formats for the sake of controlling the standard and locking up the market. At least, I don’t believe that’s their motivation. This is why there is no Apple video codec, or audio “codec,” or graphics “codec.” Apple uses PostScript (PDF) as the basis for its graphics engine rather than developing one of its own. Isn’t that pretty darn open?

No, the beef isn’t with Apple’s openness or about conversion issues, which are generally very easy to work out. Actually, one of the risk factors for data that Pilgrim leaves out is the degree of support the format has from developers. To that extent, as Gruber says, the era of being at risk by using a Mac is now over, and if anything the pendulum is swinging the other way. There are so many developers building quality Mac OS X applications nowadays, that Cocoa is well on its way to being a factor that converts users to the Mac all by itself.

On the other hand, a format like mbox is used less often nowadays. Thunderbird doesn’t use it, and neither does Outlook, or Opera mail. Eudora has a shrinking slice of the pie, and Apple Mail’s slice is rising. I’m not saying mbox is at risk, but I wouldn’t count on it being around forever. It’ll be around only so long as there are developers willing to support it, which requires customers who are demanding it. Moving from one minority platform to an OS with even smaller support–especially when the platform you’re leaving supports everything the minority platform does–seems a little odd. Especially when the platform you’re leaving is more open than it has ever been before, as Gruber points out.

So odd, in fact, that I think it can’t be explained logically. None of the reasons Pilgrim gives make any sense. I’m not arguing for Mac OS X or against Linux here, I’m just saying a switch like this takes a great deal of effort, and why turn your world upside down over a change in mail formats? Especially when all you really have to do is switch mail clients. That, I could understand. I’ve done it often enough myself.

No, what we are witnessing is the end of a love affair. And when someone falls “out of love” with a company like Apple, there are no counselors the couple can turn to for help. So they do the only thing possible… a clean break, get away from everything that reminds you of the great things you accomplished with that beautiful spouse over the last 20 years.

You write up your reasons for breaking up, and everyone but you realizes you can’t see the full picture. But certainly I, like many Mac users in a similar position vis a vis Apple, mourn the breakup and wish it didn’t have to be so. Powerful emotions can close minds as surely as any proprietary format. And unlike that closed format, prying open a closed mind is nearly impossible.