Scaling on multiple CPUs for load balancing

[Student007]

Hi Joe,

First, I love your awesome work - thanks for making this free! In your roadmap you plan replication and horizontal scaling. Does this mean you will give the ability to add multiple machines what sync their JSON-Files ? I like to do vertical scaling like Nodejs http clustering - means: Using all my CPUs. I did this my own using MapRed for mapReduce, but it would be interesting to see full clustering support by LokiJS itself. It is about concurrency of the persistence level, I cannot fork myself the easy way. Because it is the same memory (I think) it could sync inMemory and persist all in the master instance. In my opinion about how LokiJS works, I think, it should be possible to fork the insert,update,get and organize the persistence sync by the master by giving API control about the number of forks. In the case I missed something, please apologize and put my nose on it :smile: I wouldn't hang on isomorphic thoughts instead of getting much more performance by clustering. The browser has web workers for threading - are you using them (couldn't find something in the doc - oh wait - the dynamicViews are web workers maybe) ?

Btw: I love to see you changed your licensing more MIT like

My goal is to get an inMemoryDB is working within all instances of Nodejs answering a http request (load balancing). I am thinking about to write a KoaJS Middleware to load multiple instances based on numbers of idle CPUs. That way the Event-Stack wouldn't block the chaining of koa middleware longer than necessary - DB is a bottleneck. In the case there is not resource to add the features like I need for my plans. It would be nice to get an explanation about how to manage this my own (maybe setting semaphores in the app itself - bound on the events LokiJS fires).

Dan

[obeliskos]

I'll let @techfort speak to roadmap, I'll just mention that I believe memory is the biggest constraint with an in-memory database. I would guess that most platforms node, browser, webkit are limited to around 2GB available for the database to use. It's large enough that persisting it can be slow, yet small enough that queries against this fast in-memory db will probably always be fast enough.

We are not currently using web workers since loki is to work on node.js which does not have them. In theory you might use them at a user level, setting up common worker messaging to proxy insert, update, delete, and find ops to a common database.

@Techfort has explored server options for loki.js, but two options might be :

• Collection Sharding where each collection goes into its own server instance (or multiple smaller collections might be in same shard). This would expand memory limits to 2GB/collection instead of 2GB per database and improved speed depending on shard structuring.
• QueryServer/Persistence partitioning, where all inserts/updates/deletes go to both servers but find ops go to Query server and Persistence server handles autosaving to potentially large database files. In an autosave scenario, saving a 2 GB file to disk every (n) seconds might take a while and be best left to a server not responsible for query requests.

[Student007]

@obeliskos thanks for the detailed first response. That's a good idea to put the JSON to a remote server like AWS. But here comes up a new question: Does the saving only update the JSON parts in the file that have been changed since last timestamp - or is it written completely new ? I think frequently saving 2GB will make a difference here. Same as sending data over network. It would be better only to send data based on timestamp later than x. If you didn't, here is a short idea of optimization: Knowing the position of each dataset by reference pointer within the database.json would make persistence much faster. Same as saving network traffic, you only need to write a service for updating the database.json ...

About the idea to share collections: This would be a good idea if they are copies not referenced collections - isn't it ?

Or is there a way to use the reference to save memory ? It is about the imagination the hole database file is in memory when getting the collection out of it. In my opinion the gotten collection is referenced, not copied. I know from the docs there is a way to copy them (if I remember right). Writing to the same collection (if it is referenced not copied) would maybe make a concurrency situation (by clustering). I'm unsure about this point. I do not know what part of the hole (operating) system is managing this. Maybe the operating system prevents read/write access to memory when it is used by another thread.

Edit: I did one more search and found this article on medium.com where it is talked about branchResultset() what I couldn't find on your official documentation at lokijs.org. It looks like the concurrency problem is solved by merging in some magic way - isn't it ? So limited copies could be referenced by condition of requested resource by the slave instance of the answering server. After answering and performing the action on the resource, the final act would be to trigger merging.

But all the thoughts might be wrong and cause in wasting capacity. (Blowing up memory is more expensive than waiting for access to a single instance). When the process forks for nodejs load balancing, the hole app (without what is triggered within the if(cluster.isMaster){...} part ) is copied in memory. So it would be the strategy to fork the lokiJS part within the master. At this point the hole communication runs over the cluster API of NodeJS. (Btw: You could use the NodeJS-Cluster like WebWorkers ! - it works the similar way)

[techfort]

@Student007 @obeliskos I admit my knowledge on how to tackle this issue is still pretty limited and I am still researching. I'm coming down the home stretch with a couple of projects right now so it will be another week or two before I can start investigating this properly. If you want to go ahead and explore possibilities that's perfectly fine by me, otherwise we can reconvene in a couple of weeks?

[obeliskos]

No rush @techfort ...

Well 'sharding' would split the database across servers. For simplicity we might do split this by collections.

This would need an interface to orchestrate queries, directing them to the appropriate shard without user knowing details about all the shards. We have not yet built such an orchestration service.

Dynamic View and Resultset branching are not process forks since JavaScript is single threaded. Branching in for Dynamic Views and Resultsets lets you design your queries to go down different filtering paths, not execution paths, The idea being that you can temporarily retain a minimal filter to be further filtered in different ways with less performance cost.

[Student007]

Oh yes the collection based sharding blended with an clean interface and a loki-server with a socket API would make it very flexible. Maybe it is a good idea to go this way, because I assume it is right about the memory will switch every change of context on the CPU (what also will change the memory working with - and this needs time).

About the argument JavaScript is single threaded: Today it isn't true anymore. We have clusters on servers, we have web workers and we can use parallel.js. We only have a single event loop (FIFO=queue).

So I think it is a very very good idea to take the next step to make lokijs managing dynamicViews in a second thread. It would be possible to do this also isomorphic. Maybe the require('lokijs')({'multithreading': true}) could decide to require modules for the server, browser or mobile.

The question is, what cases are big enough to see it is much more performing than the current version of lokijs.

Edit: MapRed is like binary search perfect for multithreading. MapReduce by its purpose indeed is only interesting with multiple threads (and machines). A single thread MapReduce is like using a nested foreach loop. Also I have taken deeper look into the code and have seen, there are some things could be optimized for the server side and with EcmaScript 6 for the client also. Finally: The performance of lokijs could be made much better if it is setup to more actual stand of tech. (my two cents) Edit II - what doesn't mean your JS-Knowhow and work isn't awesome - IT IS !

[obeliskos]

Well the main issue is multithread isn't part of the ecma standard.

Webworkers are part of w3c standard and browser support for localstorage or indexeddb within them may be spotty across browser implementations.

I think @techfort looked into JXCore a while back. I believe they spin up separate v8 engines and communicate via a MemoryStore .

[Student007]

I agree - forgot localstorage is a DOM window object. That's not usable within webworkers. :cry: JXCore I also had in mind since they are open source. But if I remember right, they are only compatible with node 0.10 not 0.12 and that is a risk indeed. They are asking for a abstracting layer for node to be independent from V8. But I think because of the less number of core contributors it is not a base to plan for long term. But it will change node - because they are right. The problem looks similar to Tessel and their Colony-Compiler what should make a MC node compatible. Now they are creating Tessel 2 based on OpenWRT and V8 ... and kicked their own runtime.

Back to current possible optimizations of LokiJS:

• MapRed
• Server :smile_cat:

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.