AOL Redo

[qpfiffer]

The AOL file is how we record meta-information in the database. After the mmap_redo branch is merged in, we will have some new advantages give the current state of things.

Since values are now no longer stored in the aol, we can guarantee that one line in the file = one operation on the database. This opens up a lot of possibilities for smaller files, ie. by removing delimiters. The new idea is to just have repeated <size_t padded with zero><data> strings serialized into a single line one after another for a given command.

In addition to this, complaints have been raised (for some reason) over not being able to compact the file to remove any old keys or commands that are no longer necessary. So, for anyone redoing the AOL file (@zeroZShadow @Hamcha) there are a few goals in mind:

1. binary format
2. Needs to be able to survive schema changes (by schema I mean parameters to calls like ol_jar, ol_unjar, ol_scoop, etc. if they change in the future
3. Compaction command that will take the current database and serialize it out as a series of instruction to a new AOL file, delete the old one and copy it over (You could accomplished this by just iterating through the splay tree and just writing a bunch of JAR commands) This is already done.
4. All operations on the AOL file should be guaranteed, this means fsync. Always.

[qpfiffer]

Other ideas:

• Do something that doesn't involve printf (It's pretty slow)
• Dont use the fopen/fwrite/fget API, we more consistently use raw open and mmap to address files