search

rethinkdb / horizon

Horizon is a realtime, open-source backend for JavaScript apps.
MIT License
6.78k stars 351 forks source link

Offline support #58

Open danielmewes opened 8 years ago

danielmewes commented 8 years ago

We should think about offline support at some point. This was mentioned by @Rajan in https://github.com/rethinkdb/rethinkdb/issues/5239#issuecomment-167700641 .

The idea is that it will make it a lot easier to build mobile applications on top of Fusion.

As a first simpler step, we could allow mirroring certain data to the client for read-only access. In a second step, we could hook into our optimistic write infrastructure (that we still need to build, https://github.com/rethinkdb/fusion/issues/42) to allow for syncing back writes once the user gets online again.

dalanmiller commented 8 years ago

Would this also require resumable changefeeds @danielmewes?

danielmewes commented 8 years ago

@dalanmiller That's a good question. I think this depends on how the API is going to look like, and how big we expect the offline data to be.

It might be acceptable to just stream a completely new set of results upon reconnection. We could additionally generate a diff in the client library. The client would retrieve a complete new result set from the server, and then would compare it to what it currently has cached. Then it would emit events only for the changes.

danielcompton commented 8 years ago

Even if resumable change feeds were present, I think you'd still need to handle the failure case as the 'default', and if the change feed is able to be resumed choose that as a nicer way to reconnect. Resumable changefeeds still have a finite buffer after which they will need to close too, and there could be other reasons why resumable changefeeds don't resume (server restarts are the one that comes to mind).

deontologician commented 8 years ago

Related to #358

COLABORATI commented 8 years ago

yes, pouchdb like offline sync is absolutely needed in 2016 for any mobile app platform, please work on this so it can be done with rethinkdb, thanks!

nevf commented 7 years ago

Progressive Web Apps and offline first are gaining momentum. There is real need for fully fledged database capabilities in the Browser that can be used offline, with sync + conflict resolution back to the server when a connection is available again. This could be days or weeks later.

By fully fledged I mean all CRUD operations, so from the user's perspective the app has all the same functionality whether they are online or offline. The Browser database world is unfortunately constrained to IndexedDB, so we need to build on top of that.

My own use case is to enable Clibu the Knowledge Base Web App I'm developing to work offline. Right now it uses MongoDB on the server along with realtime websocket push of updates to the Browser, but it has no offline capabilities. The only option I can see at this point is to use something like Dexie.js along with it's Syncable module, but I can see that being a lot of work. An end-to-end solution such as RethinkDB + Horizon suggests itself to be far better.

I continue to be surprised that no one has yet come up with a modern NoSQL package that accomplishes what PouchDB and CouchDB do, clearly there is a now more of a need than ever.

paulocoghi commented 7 years ago

I continue to be surprised that no one has yet come up with a modern NoSQL package that accomplishes what PouchDB and CouchDB do, clearly there is a now more of a need than ever.

This is the point.

franciscolourenco commented 7 years ago

Hoodie

A generic backend with a client API for Offline First applications https://github.com/hoodiehq/hoodie

nevf commented 7 years ago

@aristidesfl Have you actually used Hoodie and if so how did your project work out.

I personally find the Hoodie website and documentation quite difficult to follow, with no clear explanation about what it actually is.

franciscolourenco commented 7 years ago

@nevf I only played with it but the experience was very good. The website however has seen better days I agree. The docs are based on https://readthedocs.org/.

If you jump into their chat you will receive awesome support.

tohagan commented 7 years ago

If you're wanting offline sync support you might want to have a look at Superlogin. It's based on Couch/Pouch but adds Node/Express/Passport to do authentication and a feature complete user account management API that can be used side by side with Passport's OAuth for Google/Facebook/Twitter etc. Great documentation & demo. Includes a nice client side API and an AngularJS wrapper. CouchDb supports Javascript validation rules that are stored and replicated as part of a "design document". When offline, PouchDb has a plugin to execute these. They get re-executed when your objects are replicated to the CouchDb server.

Like RethinkDB, CouchDb supports a realtime changes feed. Changes filtering is slow in CouchDb 1.x but CouchDb 2.0 addresses this and also adds a decent query language. RethinkDB uses a master-slave architecture while CouchDb is multi-master (so you can update any node including an offline one!). It's uses document versioning to detect and partially resolve update conflicts between nodes. If you only use delta documents and then aggregate them into map/reduce indexed views you can frequently avoid collisions by avoiding all document updates.

juliobetta commented 7 years ago

Couldn't agree more with @tohagan. I've been using pouchdb and superlogin for a while... CouchDB/PouchDB/Superlogin are really awesome!