No Mongo Indexes

[bassdread]

Currently no indexes are installed for any of the Mongo collections. Best practice would be to setup them up on collection creation.

My guess would be to at least index both token and uuid fields for the events collection to ensure fast searches as the collection expands.

[megastef]

Could you advise where I can find the database module? E.g. to replace mongo with something better?

[chrismatthieu]

Hi Stefan,

Thanks for reaching out to us! We would welcome your help in creating a database adaptor with support for multiple datastores. We recently made Splunk, Elastic Search, Mongo, and Redis optional and fall back to NEDB for file-based data storage and memory for session storage. Our goal was to make it super simple to deploy private SkyNet clouds on Raspberry Pis, Intel Galileos, etc. Our SkyNet service is deployed in Amazon EC2 so we are thinking about adding ElasticCache (Amazon Redis) and Amazon's Dynamo NoSQL service to the mix. Config.js would control which datasource was used for each deployment. I think that it would be cool to add CRATE to the mix as well!

The main database section is https://github.com/skynetim/skynet/blob/master/lib/database.js but most of the files in lib (https://github.com/skynetim/skynet/tree/master/lib) also interact the DBs differently. It would be nice to DRY up the lib files to use a database adapter to eliminate the if/elses.

Our ElasticSearch updates are all handled here - https://github.com/skynetim/skynet/blob/master/lib/logEvent.js

Let me know if you have any questions.

Thanks! Chris

[chrismatthieu]

While we do not have migration scripts yet, we have added indexes to our production mongo databases and have seen a significant performance increase. It also helps that we have been moving more and more realtime mongo lookups to redis.