Improve performance and stability of /bulk endpoint, as well the architecture of the code. Readability is poor right now.
Stability:
Operations are currently not atomic and if one fails, it's reported as an error, as expected, but others will be successful. The integrity is compromised.
Each transaction should be saved in a log and kept if something went wrong
Performance:
We need to use the bulk endpoint from elasticsearch. It will be way faster than the current implementation. It's also easier to deal with errors. Since we generate IDs, everything can be flushed in the same transaction.
Architecture:
The code grew in size, let's put all managers into a new directory. server.coffee should have less logic in it and should only be a router.
draft
goal:
Improve performance and stability of
/bulk
endpoint, as well the architecture of the code. Readability is poor right now.Stability:
Operations are currently not atomic and if one fails, it's reported as an error, as expected, but others will be successful. The integrity is compromised.
Each transaction should be saved in a log and kept if something went wrong
Performance:
We need to use the bulk endpoint from elasticsearch. It will be way faster than the current implementation. It's also easier to deal with errors. Since we generate IDs, everything can be flushed in the same transaction.
Architecture:
The code grew in size, let's put all managers into a new directory.
server.coffee
should have less logic in it and should only be a router.