Ijod

[temas]

This implemens ijod for core synclet results.

A few of the highlights now are:

• JSON data is tracked with a murmurhash3 to ensure we don't reprocess as much
• Writes from the synclets are batch processed leading to a large performance gain.

There are currently two primary TODOs before looking to merge.

• Look into data access patterns such as /getCurrent
• More testing!

Please test this branch and see how it feels.

[temas]

getCurrent is unlost and functioning.

[temas]

This is now ready to merge and start testing on our sandbox. I'd appreciate any final reviews before I pull the trigger.

There is a verification tool in the Ops directory called checkCurrent.js. It is ran such as: node checkCurrent.js localhost and to test against ijod use: node checkCurrent.js localhost ijod. These are using the same calls, just different file naming. The list of synclets can be adjusted at the top of the file for testing based on available data sets. The results are put in a verification directory from where it's ran. There seems to be a small issue with trailing \n missing sometimes, but all of the data is the same, verified with sha1 sums and diff.