Review possible use of orbit.js

[silverbucket]

https://github.com/cerebris/orbit.js

Seems like it could be worth a look for replacing our syncing internals. Maybe not also. But worth a review by others.

Seems to have searching and syncing built in.

[raucao]

That sounds like a great idea!

Unfortunately orbit.js seems to be using localStorage at the moment. Or did I miss something?

[silverbucket]

Correct, depending on the architecture, it might be simple enough to add IndexDB support, using our code as a reference.

[silverbucket]

Oh, and just after I posted that I see this issue: https://github.com/cerebris/orbit.js/issues/6

[silverbucket]

@dgeb We're considering using orbit.js for the internals of remotestorage.js, which currently has a home-brewed syncing solution that we're still trying to work out issues with. I was wondering if you had any thoughts or comments which might help us to make a more educated assessment?

[dgeb]

@silverbucket Seems like a great use case for Orbit!

Orbit currently contains a set of three compatible sources which all implement the same actions in their Requestable and Transformable interfaces. I'm considering separating these sources (by namespace and perhaps repo) into a "stdlib" of sorts.

I fully expect the stdlib itself to grow as we add an IndexedDB source and others. However, the real potential is in creating sources such as yours which will gain compatibility with the rest of the lib simply by implementing both/either Orbit's Requestable and Transformable interfaces.

There will be a lot of churn in the Orbit repo as I refactor and document it in the coming weeks. It will probably be split into multiple repos, moved to its own org, and get its own site where docs can be easily browsed.

It's up to you whether you want to wait out the churn or dive in now. Either way, let me know how it goes!

[michielbdejong]

afaics there are a few things we have now (IndexedDB, our Merkle-tree-like sync strategy, refresh intervals, conflict events, controlling the number of concurrent REST requests, dealing with Auth failures, resuming sync after a page refresh, different caching strategies per sub-tree) and which i don't see from a quick scan of Orbit's API, so i don't really see a way to use it in our library.

but Orbit in itself is a cool project! @dgeb did you check out PouchDB as well? it implements CouchDB replication between client-side and server-side

[dgeb]

Although Orbit doesn't provide many of those features out of the box, I've tried to take them into account in its design:

• An IndexedDB source is on the roadmap.
• Custom sync strategies can be implemented by overriding hooks in the TransformConnector.
• Refresh intervals could be placed in a custom source, which could control caching of data and even automatically retrieve fresh data on a schedule.
• Auth failures will currently prevent requests from completing.
• By sync'ing with an IndexedDB or LocalStorage source, a data cache can be maintained locally that can be sync'd remotely after a refresh or the user comes online.
• An Orbit PouchDB source could be developed.

Orbit will soon provide compatibility across a wide range of data sources and frameworks. For instance, once I build Orbit hooks into Ember.js, any Orbit data source will gain compatibility with Ember for "free".

Of course, Orbit's very young, and I totally understand wanting to wait it out to see how it matures. Anyway, thanks for considering it!

[raucao]

Sounds great. Maybe it doesn't make sense for us to commit to something right now, but I haven't seen a project that would fit better so far. I think our storage and sync code should be separated from remoteStorage logic at some point in the future, especially in light of the fact that we decided to implement 3 different backends already.