Closed lahwaacz closed 9 years ago
I've made some experiments with BlitzDB and either I've missed something important or document databases are not really good for this big data yet. Anyway, custom JSON database seems to have the best effort/performance ratio, so this is being implemented starting with 78ac098260f02eabc0e6e61d014d9b5c1060f23a.
Some scripts fetch big data which is not changing often (or ever), so repeated queries are slow and unnecessarily increase the load on the server. Caching the results of the queries in persistent storage would be solution.
For the storage backend I'm interested in a document-oriented database with easy Python interface. Some considerations:
Some alternatives are mentioned here: http://stackoverflow.com/a/5835099/4180822