moiri / streamix

Root repository for the Streamix project
0 stars 1 forks source link

Dump Data to File Before Sending to DB #11

Open moiri opened 5 years ago

moiri commented 5 years ago

Currently, in smx_mongo data is aggregated in memory during a configurable period of time and as soon as this time has elapsed, the data is stored to the database.

If the program crashes at some point, some data is lost (and this might be a lot, depending on the configured period). In order to remedy this an additional net could be introduced in parallel to smx_mongo which dumps the data to a binary file which is used as a circular buffer.