use IndexDB to store data locally

[bblfish]

Would be very handy to increase startup time, and also to allow different components potentially running in different tabs but served by the same server to work easily together without needing to fetch all the data again each time. See issue #12: "component selection mechanism"

Requires one to write such a store, but it should be a lot easier in ScalaJS than in another language.

[bblfish]

See discussion in issue 1 of scalajs-rx-idb

Here is a summary from the discussion:

I think we agree that Rx and Actors are very different tools. So different in fact that one can build Rx on top of Actors :-)

• Actors do not come out of the box with backpressure support - in fact out of the box they are more likely to drop messages - but they can be used to support backpressure by sending messages back to the sender notifying him of overload ( It's tricky to do this right but it's done in the implemenation of akka-streams ).
• Actors are low level constructs to make it easy to create resilient applications, because the actor hierarchy makes it possible to stay responsive and to recover in the event of failure.
• An important difference is that Actors are Message-Driven whereas Rx is Event-Driven.
• Both are asynchronous and non-blocking.
• Actors contain type safe code, but externally hide types ( ! is a function Any => Unit ) whereas Rx is type-safe functional code that is easy to compose.
• Actors are built for Location Transparency, I don't think Rx is
• Actors make delegation easy.

So these two tools are very different, but can presumably be used together, either by building Rx on Actors or simply by having actors return Rxs .

For instance working with IndexedDb is basically working with streams of data + you need backpressure because the subscriber/consumer of data can read them with different pace then IndexedDb yields it and the same applies for writing data to IndexedDb. Consumer is setting the pace of reading and IndexedDb is setting the pace of writing... If you used akka it would just overflow buffers because it has no backpressure ability.

That makes a very good case for backpressure. We may also want to have WebWorkers open connections to the web in read and write. ( I am not sure how well backpressure works over XMLHttpRequests though ).

Regarding lock starvation, I'm talking about IndexedDb transaction handling, in the sense that you don't want to open new transaction unless you closed previous one when R/W the same data. It might happen when you are not careful when R/W from/to stream, you must respect onComplete/onError event signaling end of previous transaction to initialize new one... Otherwise it usually leads to huge throughput decrease in IndexedDb caused by something like "lock starvation"...

This is where I am thinking Actors and Rx could work together (let's forget for the moment about what may or may not belong in the browser). What you have is a problem where a number of different parts of the code need to be aware of which transactions are happening: ie. they need to synchronise on these transactions. Since we don't and can't use Java like synchronise tools on a global object, and furthermore we may want to delegate access to the IndexDB to WebWorkers which means we can't rely on a synchronisation object, and we also want to be reactive, we need to find a reactive method of synchronisation. This is provided by an akka actor, since messages to a certain DB can be funnelled through the actor which can only do one thing at a time, wherever it is located. So I can imagine a role here for Actors returning Rx objects.

Also: In order to reduce lock starvation would it make sense to have a lot of little IndexDB databases? In my use case I am fetching Linked Data off the web, following links around the web. Would it make sense to have on DB per resource on the web?