use indexedDB/WebSQL

[calvinmetcalf]

instead of local storage you should used indexedDB (and WebSQL for older browsers) better size limits and faster

[dgeb]

I agree that an IndexedDB source would be quite useful and it's definitely on the roadmap.

However, it's not an either / or situation, since any number and type of sources can co-exist in an application. Local storage is useful in some scenarios too.

[michiel]

I've started writing an IndexedDB source for orbit. As it's on the roadmap I was wondering if anyone else was working on this. (ref: https://github.com/michiel/orbit.js/tree/indexed-db )

[dgeb]

@michiel - Although it's on the roadmap, I haven't personally done any work on an IndexedDB source yet (I'm currently focusing on an ember-orbit integration library). Thanks in advance for your contribution!

[JannikStreek]

I am kind of in need of the indexeddb+websql sources, @michiel can I help you with the implementation? is somebody already working on the websql source? If not, I would like to do it!

[michiel]

Sure, indexeddb is still on my todo list. If you want to help out, fork my branch at https://github.com/michiel/orbit.js/tree/indexed-db . It's a work in progress and is heavy on the log statements. I run (failing) tests in Firefox and Chrome. PhantomJS doesn't support IndexedDB.

The files you want to look at are,

• lib/orbit_common/indexed_db_source.js
• test/tests/orbit_common/unit/indexed_db_source_test.js

They're based on the jsonapi one and my plan is to have the indexeddb source pass the same tests as the jsonapi tests (async interface) so indexed_db_source_test.js should look very familiar.

A few notes,

• one of the first things that's different is that the source.init() call takes a callback argument as initializing the db is an async operation.
• all interaction with the db happens through functions prefixed with _idb
• all async operations (except for init) are wrapped in Promises

As for WebSQL, for what I have in mind I'll need that at some point too, at least as a PoC. I was thinking of seeing how far IndexedDBShim ( https://github.com/axemclion/IndexedDBShim ) would go for that as it seems to support all the calls I'm currently using.

[JannikStreek]

Sorry, took me longer than I thought to get finally started. @michiel Before I start adding PRs, could you elaborate on why you inherit from the source object, instead just using the memorySource object? If I see it correctly, you could just listen for the didTransform event in your implementation. In addition, you need an internal cache for the indexedDB anyway, so just extending memorySource would be easier. Or did i miss something?

[opsb]

I'd expect indexedDB to not have an internal cache, wouldn't you just wire up an in-memory source in front of it? You could always package up a combined adapter that included both indexedDB and an in-memory cache but I think it would be good to have the standalone indexedDB source.

[JannikStreek]

@opsb The internal cache is not really a requested feature by myself, it is more an implementation requirement if i see it correctly. If you have a look at the source.js, every resource has an internal cache.

[opsb]

@JannikStreek I hadn't realised they all include an internal cache, will have to take a closer look.

[dgeb]

@opsb @JannikStreek The internal cache is important for sources that might be both sync sources and targets. It allows for bi-directional sync'ing of transforms that don't infinitely loop (source a -> b -> a -> b). It also makes sync'ing more efficient by sending only data that has changed.

For simplicity, I added a cache to all OC sources at first. However, I have plans to make it optional, since it can obviously be expensive memory-wise and shouldn't be used unless needed.

[dgeb]

I've created an IndexedDBSource in a new repo - orbit-indexeddb.