Remove Main Log, add Journal, make shards durable

[jankotek]

Main Log is suppose to be central component, where all data are stored and which provides durability. Shards are suppose to provide faster lookups.

However there is design flaw; if all data are in single Log, it becomes large and impossible to compact/clean in reasonable time.

It also leads to data duplication. Data are stored twice; in Main Log and in Shards.

So we need to:

• remove Main Log
• make Shards durable
• provide temporary durable Journal for storing N last updates
  • do not store all data in Journal, just Updated data
• Sharding Task takes data from Journal and distributes them into Shards
• once data are durably written to Shards, Sharding Task deletes Journal
• Modify Key Lookup to first look into Journal, than Shards

[jankotek]

Fixed in 0.2 release