feed and validation basically did the same things, processed queues of messages.
Though, I had done validation a weird way... it kept everything in one queue,
and then only worked on the head. I've updated this so it keeps a queue for each id.
it keeps the latest message from each feed in memory, which makes validation really easy.
It still needs to keep an queue, in case multiple writes come in before reading the current state has completed.
Feeds (local writes) have been greatly simplified. instead of implementing another queue,
it just pushes the create function onto the validation queue, creating the actual message at the last moment (oops, just did a monad ;)
TODO: If there are huge numbers of feeds (thousands), clean up ones that havn't been used in a while.
feed and validation basically did the same things, processed queues of messages.
Though, I had done validation a weird way... it kept everything in one queue, and then only worked on the head. I've updated this so it keeps a queue for each id.
it keeps the latest message from each feed in memory, which makes validation really easy. It still needs to keep an queue, in case multiple writes come in before reading the current state has completed.
Feeds (local writes) have been greatly simplified. instead of implementing another queue, it just pushes the create function onto the validation queue, creating the actual message at the last moment (oops, just did a monad ;)
TODO: If there are huge numbers of feeds (thousands), clean up ones that havn't been used in a while.