I’ve gotten a few e-mails now from people asking me why I have chosen CoreData over a standard mail format like Maildir. Kiwi will support Maildir for importing and exporting your mail (via Libetpan), but it will not be using it as it’s primary storage method. Here’s a few of the pro’s and con’s for each:

CoreData

Advantages

Fast

Atomic Commit

Manages object graph

Simplifies implementation of multithreading

Allows for easy integration of added metadata like tags

Disadvantages

Format is application specific

Makes Spotlight integration more difficult

Maildir

Advantages

Standard mail storage format

One message per file makes Spotlight integration easy

Small memory footprint

Disadvantage’s

Slower

Makes persistence code much more complex

Makes implementation of the interface much more complex

It’s impossible for me to ignore the speed and ease that CoreData brings to the table. While there are definite advantages to using an open standard like Maildir, CoreData is going to make it easier implement more innovate features. In a way the format for Kiwi is not proprietary, since the application will be open source and everyone will have access to the schema for the SQLlite database. So hopefully, CoreData will be the right choice and will allow us to build a better app faster.

Ideally in Kiwi, everything would be pulled from the server during launch, but this limits Spotlight integration and it makes offline reading of e-mail impossible. I originally toyed around with storing e-mail in the MH mail format because it’s an open format and every message has it’s own file. Persistence is never easy, especially when people demand stability, and implementing the MH format is no exception. Knee deep into writing file I/O code, I took a step back and rewrote with CoreData.

Previously, the interface fed off of instance variables in objects via bindings. It was clear that this was making persistance and multithreading more complex than necessary. So I packaged all the code together that performs IMAP calls and sat it on top of CoreData. Now, instead of writing retrieved IMAP data to local instance variables, it writes directly to the CoreData store. The interface sits on top of a copy of the CoreData store and feeds off of it with Cocoa bindings.

If I had used the MH format, I would have had to manage all of the relationships between Folders and Messages and more importantly, I also would have to insure consistency in the event of crash. CoreData has been a huge help in this regard, it maintains the referential integrity for me and it keeps changes in memory until they are committed. Now users of Kiwi can be confident that their data is stored in a SQLlite database that is consistent across sessions.

Moving from a graph of NSObjects in memory to CoreData has not been without trial and error, though. For about the past day, I’ve been trying to figure out why one table hasn’t been displaying data that exists in the database. I double checked all of my connections, I tripled checked my methods and still, the table was displaying a lone and empty cell. Finally, today I figured it out: I had forgotten to switch the controller in Interface Builder from Class Mode to Entity mode. I made the simple switch, and my tableview (thankfully) came to life.

I have to say that these new technologies, like Bindings and CoreData, solve problems but they also introduce yet another learning curve for developers on the Mac platform. In the end however, I think CoreData is going cause alot of developers out there to take a second look at their persistence code and wether CoreData and Bindings can help them spend more time on other features.

It’s been a long time since I’ve been as excited and motived as I am now to develop an app. I’m revved, so much so that I’ve been glued to my computer for the past few days and it’s sure payed off. Kiwi now has tabs; e-mail is been rendered using CSS, a sudo-preprocessor and WebKit; Major performance improvements; and the Interface is getting cleaned up. Now that I’ve sketched out the app, it’s time to start filling in the details that are going to make this thing great. And I probably shouldn’t say “I” anymore, a few of my friends have jumped in and are helping out now. One of my friends from the University of Illinois, Bob Van Osten, has joined in providing alot of added interface code, including the tabs and many of the interface tweaks. He’s also actively working on getting SMTP up and running, which i’ve neglected completely so far. My friend Chad Weider, has been assisting in writing up regular expressions to help process e-mail. Here are more screenshots, the interface is a little rough right now because of the tabs, but we are working on cleaning that up.

Kiwi is coming along quite nicely now. Much of my progress is thanks to Cocoa bindings, which have made glue code nearly non-existant. Bindings are fantastic, but they come with a very steep learning curve and I feel like I’m just beginning to grasp them now. Anyway, here’s a bit of what I’ve gotten done in my app:

• Account settings are now configurable and SSL is supported
• Subscribed folders are listed on the left in an outline view
• Unread status is parsed for messages
• Sender information is parsed
• MIME decoding is supported so e-mail output is greatly cleaned up
• Connection support has been improved, resulting in a huge speed up
• There is a toolbar and a toolbar item that loads e-mail, finally!

Of course a picture is worth a thousand words, so take a look at how far it’s come:

So what do I have planned once basic e-mail support is complete? I hope to add tab support and e-mail rendering using CSS and Webkit for very customizable output. Stay tuned.

I’ve been using the wicked cool Textmate to do programming for the last week. It’s one of those apps that after every launch you find a new little feature that draws you in even more. Each little feature makes you more productive and your workflow more streamline. Earlier this week I had one of those moments with Textmate when my friend Chad mentioned that when he was doing work in Eclipse this summer, there was this great feature that would allow him to add comments like: //TODO This is something I need to do And at the bottom of the window there was a pane showing him what he marked as to do for that file. After, hearing this I played around with Textmate looking for something similar, and there is! If you add comments like shown above to your files, and then run the command TODO, it will display a nice window with hyperlinks to all the things you marked. Very cool: Take a look below:

I’ve been doing some hardcore hacking on my pet project, an e-mail client named Kiwi, and what you see above are it’s first breaths of life. It’s not much to look at, but it’s significant progress.

I am finally able to load the inbox and display messages when they are selected in the rightmost pane. Don’t mind the ugly e-mail output right now, I’ve yet to do any parsing of the headers and the MIME data.

Here’s a cool trick that converts man pages to postscript and opens the file in Preview. Add this to your .tcshrc file in your home directory:

alias preman 'man -t !^ > /tmp/!^.ps; open /tmp/!^.ps'

Then to open man pages in Preview type:

preman NAMEHERE

I discovered the incredibly useful unix program screen just recently. It allows multiple terminal sessions running at once and it provides easy access for switching between sessions. To run the command type: screen Once screen is running you can do Control-AC to create a new window and use Control-A followed by a number to switch to that session number. And the best I’ve saved for last, if you close your terminal session and want to get back to the screen you were working on, just type: screen -r Magically, screen will come to life just as you left it, which makes it great for running on remote servers.

Due to my dissatisfaction with existing mail clients, I’ve been slowly grinding out a simple IMAP client. There was no way I was going to write my own IMAP client, so I scoured the web and came across a few libraries. So I wanted something that was clean, fast and written in C, C++ or Objective-C and ideally thread safe.

C-Client C-Client was written by the man at the University of Washington who invented IMAP4rev1, so you can be sure it implements the RFC fully. However, the code is downright atrocious, it’s a mess of spaghetti code and undecipherable documentation. There are two programs which are samples to help aide in understanding the library. One of the samples is an old mail program written for NeXt and the other is Pine. If you haven’t looked at the Pine code, I urge you not to, it burns the retinas. So despite C-Client implementing the spec perfectly and being a very fast library, I found myself unable to understand the wild style that the library was written in.

JavaMail The JavaMail API’s look really clean, but I was leery of implementing a core part of my e-mail app on the Java-ObjC bridge. In a past project, LaserLine at UIUC MacWarriors, I was forced to use the Java-ObjC bridge and was not pleased. I found it a hassle to deal with and a slow alternative to native C code.

Camel Camel is the mail library that rests under Evolution and it’s designed to access a number of different kinds of data stores. It’s written in plain C and relies on a number of data structures that reside in the underlying Gnome libraries. The libraries that it does rely on are non-graphical so getting them up and running wasn’t a big deal. The code base is massive, with support for an amazing array of data stores. Some people are very leery of the IMAP support inside the library, apparently it works but it’s not exactly thread safe. Camel really is a mountain of source, and the fact that they pretended C is object-oriented when writing the library doesn’t help things. What really made me drop the ball was this:

Even with all of it’s problems, this is the most stable and featureful remote backend in existence. It just isn’t very easy to maintain. I should really add more detail here, but I just don’t care - I barely understand the code either, and I rewrote it anyway. Maybe others can update it. Source

When the guy who created the monster says he doesn’t understand it, I’ll look elsewhere.

Pantomine Pantomine was the only Cocoa/Objective-C IMAP library that I found, which is a huge plus for this library. But it’s slooooowww and no one has done any work on the project for a number of years. One of the objectives of my mail client is for it to be blazing fast, so Pantomine is a no.

LibEtPan LibEtPan is the best IMAP library I have found. It’s fast, written in pure C and it doesn’t require really any dependencies. The function names are clear and the data structures are intelligible. There is documentation provided with the library but it’s lacking in a number of respects, but the code is so clean that many times you don’t even need the docs. The implementation is very robust with support for high level access, so that different mail drivers can sit underneath. For example you can use the high level function to talk to either IMAP or POP3 without having to change any code, you just change the driver. The available functions for the high level stuff is limited because it has to be the intersection of both IMAP, POP3 and the other protocols it implements. The lower level stuff is very clean as well, and implementation specific. For example, in my project I am using the lower level IMAP drivers because I don’t intend on supporting any other protocols and I get the niceties of IMAP. This library is in active development and the developer is very responsive to questions, if you’ve got a need for a mail library I highly suggest this one.

I found this in the ACM office here at the University of Illinois. Picture thanks to Nicholas Riley.

I’m a member of a number of online mailing lists, which I have an e-mail address specifically for. I’ve connected to my e-mail mailing list account through POP3 and I then sort them using a e-mail client rules. However, i’ve gotten tired of waiting for hundreds of messages to downloaded everyday, now I have moved all of my rules server side and I’m using IMAP.

On my Dreamhost account, I have setup filters to direct all of my mail to different folders in my IMAP account. Mail for UIUC stuff gets filled into a folder, ACM stuff has a folder, Mac OS X Talk has a folder and etc. I have all of my e-mail being funneled into one account but parsed into seperate folders. So if I need to check e-mail anyway, I can just connect to my IMAP account and everything is at my fingertips, including my archive of e-mail. I’m hoping to eventually condense everything down into one e-mail address, sweet indeed.

NetNewsWire Pro rocks my socks, so when I went looking for a desktop tool to tie into WordPress I figured MarsEdit would be it. While, MarsEdit is a cool app, it just doesn’t feel as complete and polished as NetNewsWire. What I was really looking for was a tool to write on my blog without fussing with HTML tags, but MarsEdit still made me mess with HTML formatting. Sorry, but hand coding HTML is too old school for me. Why should I have to mess around with HTML tags?

So I looked around the market to see what else I could find, and I stumbled across this little gem called Ecto. If you are a blogger and have not seen this app, I recommend you check it out right now. It makes blogging painless, you can categorize things, format posts, upload pictures, insert iTunes and Amazon links, it plain rocks.

Oh, and another thing, it’s shareware. And I have to say that more often I lean towards shareware/indie developed software than their commercial siblings. It feels really good to “stick it to the man” in a sense and support the little guys out there. Plus, things like blogging tools are a little to cutting edge for the major companies to have caught on as of yet.

Check out Ecto, combine that with DreamHost and WordPress and you’ve got yourself a kick ass blogging platform.

Entourage 2004 is stuck in the pre-OS X interface days. Although the application has received some overhauls, it still feels clunky. On OS 9, Entourage was best of the breed. I lived and died by Entourage, no other app could hold a candle to it, but Entourage has not matured like I wish it had. It’s now slow, buggy, and maybe worst of all: hideous. I decided that it was time to leave behind the dinosaur and find something new. My first stop was Mail.app.

My biggest gripe with Mail.app is that it does not have a widescreen/3-pane view like Entourage does. I figured that it might be possible to modify the Nib file and move the preview pane. After a short hacking session it became evident that it wasn’t going to work. The splitter between the preview and the message pane is drawn via code. If the splitter is switched from horizontal to vertical it gets all messed up and the window doesn’t display correctly. So I was out of luck with Mail.app.

Then I remembered hearing about Thunderbird supporting a widescreen view. Since I have been very impressed with the polish and usability of Firefox and I decided to give Thunderbird a shot. And I have to say, it seems like I made the right decision. I am very impressed; it’s incredible the quality of software that the Mozilla foundation has been churning out lately. The application is cross platform but it feels very OS X native. While it may not have the same level of integration as Mail.app, it really holds it’s own. I’ve found Thunderbird is very very fast at downloading new message, and order of magnitude faster than Entourage. All of my mailing lists are on a Pop3 account so that increased speed up is a big win for me. The threading support is nice, oh and did I mention it has a widescreen view.

But Thunderbird is not perfect. I would really like to see support for looking words up in a dictionary via a pop-menu, integration with Spotlight and the Address Book. Besides the above, it’s a great client. It’s prettier than Entourage and more heavyweight than Mail.app, a perfect fit for me. Here’s a screenshot of my new and old setups. Thunderbird vs. Entourage

Right now I am running:

• Safari
• Xcode
• Terminal
• Thunderbird
• Adium
• NetNewsWire
• iTunes
• OmniGraffle
• ecto
• Textmate
• InterfaceBuilder
• Quicksilver

What apps are you running?

1. On the local machine run this (when it asks for a passphrase just hit return):
   
   ssh-keygen -t rsa


2. Copy the newly created file from the local machine to the remote machine and append the key to the correct file:
   
   scp ~/.ssh/id_rsa.pub remotemachine:~/.ssh/new_key
ssh remotemachine
cat ~/.ssh/new_key >> ~/.ssh/authorized_keys
rm ~/.ssh/new_key

What a steal deal, $7 for a whole year of hosting over at Dreamhost. When it took my old host, Hostcentric, over a week to rebuild and restore the server that my website was on I decided it was time to switch. Based off of a friends recommendation, I signed up with Dreamhost and I’ve been very impressed. Want to setup a blog? Then sign up with Dreamhost. They have a one button install of WordPress that kicks ass.

And oh yea, about that deal. Sign up for the Crazy Domain Insane plan for a year and enter the promotion code 777. There you go, $7 for the year.

There is a cool article at MacDevCenter on writing Spotlight plugins. While it’s great that 3rd party developers are writing spotlight plugins, I do have a few qualms with some recently released plugins. For example, the OmniGraffle spotlight plugin includes an attribute called Number of Shapes. How is this ever going to be of use? How is knowing the number of layers going to help filter a search? It beats me. Maybe someone out there has a clever use for it? Also, OmniGraffle includes an attribute named Number of Layers. Uh oh, OS X already has an attribute Layers that is supposed to be for the number of layers in a document. I think Omni has just created a duplicate attribute by mistake.

So I hope:

1. Developers include attributes which are going to be of use
2. They do not create duplicate attributes

For while now, I’ve been dissatisfied with Mac OS X backup apps. There are too many apps that claim to backup, when all they do is sync. In my book synching and backing up are two different things, yet the market is filled with shareware apps that synch. The apps that do perform incremental backups are limited, like where’s the search? Worse yet they are cumbersome to setup and restore from. So for the last couple of years I’ve stuck with Retrospect, which is a piece of garbage (would someone please tell me why it breaks with every major OS update?) What is my solution to this? Well, try writing my own app, of course. My goal was to design a backup app that would have a Mac feel to it (why do none of the apps have searching? Retrospect’s doesn’t count, it sucks too much to be called searching) A few friends and I have been working on putting something together until we saw this. Damn, that’s pretty much exactly what I had in mind, damn you Apple. So for now I’m holding off on development and I’m going to see what Apple releases.

Lately I’ve been doing file system work in cocoa and I was using NSFileManager, but that quickly got out of hand as NSFileManager is lacking in regards to symbolic links. So, I factored out the often used code and created a class which uses FSRefs internally. However, I’ve since had a problem: I don’t want to resolve symbolic links but almost every method out there automatically resolves them. I’ve been able to work around many of the methods that resolve symbolic links by using Carbon, but there are still a few pesky problems. The biggest one is: How do you determine your read/write privileges for a symbolic link and not what the link points to? I’ve played around with a number of different ways to get the access privileges, here’s of taste of what I’ve tried:

[NSFileManager isReadableAtPath:] This is no good because it automatically resolves links

access() function Also a no go because it automatically resolves symbolic links.

FSGetUserPrivilegesPermissions(); This one seems fairly promising, if only I could get it to work right. It’s a function from MoreFilesX that goes out to the files/folders catalog info and retrieves/creates a userPrivileges int which contains what the current user has access to. However, it is not reliable whatsoever. It frequently tells me that directories which are marked no access are read only? So maybe it’s something I’m doing wrong, and it looks to me like I’m not the first to experience this. So what is going on? Here’s a snip of code:

OSErr    err;
UInt8    userPrivileges;
bool     isReadable;
err = FSGetUserPrivilegesPermissions( &ref, &userPrivileges, NULL );
if ( err != 0 )
{
   //error stuff
}
isReadable = !( userPrivileges & kioACUserNoSeeFilesMask );

Any one know another way besides reading the file permissions and checking if the user is part of that group and etc?