Make storage backend pluggable

[marshall007]

We've tried to solve a lot of the same problems in rethinkdb/horizon, but the approach you've taken in rxdb is simpler in a lot of ways. I'd love to work with you a bit on this to see if we can make your client compatible with Horizon server and/or RethinkDB. Given that RethinkDB (and thus Horizon) supports changefeeds, I think it would be as great fit as a storage backend for this project (reactive, replication, offline-first).

@pubkey let me know what your thoughts are!

[kulshekhar]

:+1: Making the storage backend pluggable would be massively helpful

[pubkey]

@marshall007 I have it in mind an will write at the weekend. I think its a great idea and we should work on it.

[nevf]

@pubkey First let me say it is great to see work being done on "offline first" Browser databases, it is desperately needed.

Given that rxdb uses pouchdb am I correct in assuming that the server needs to be running CouchDB or PouchDB server?

For my use I want to use a modern NoSQL database on the server, one that can use dynamic queries, not baked in, the way CouchDB works. More specifically I want to use MongoDB or possibly RethinkDB.

Please see my Issue at RethinkDB for more details.

Your thoughts please?

[ewnd9]

@nevf

one that can use dynamic queries, not baked in, the way CouchDB works

What do you think about pouchdb-find (which is used in this project) and find from CouchDB 2.0?

[pubkey]

So here my strategic thoughts on how to implement sync between RxDB and a rethinkDB-server:

1. We need to implement conflict-strategies, described here
2. We need a RxDB.plugin which has the same conflict-resolution as rethinkDB
3. We need an RxDB-adapter-plugin which takes a horzion-instance and transforms the changes and events in a way that they can be handled with RxDB.

The usage would look like:

RxDB.plugin('rxdb-conflict-rethink');
RxDB.plugin('rxdb-adapter-horizon');
const db =  await RxDB.create('mydb'); //....
const col = await db.collection('mySyncedCollection', mySchemaWithRethinkConflictStrategy);
col.sync(horzionInstance);

@marshall007 @nevf What do you think?

[marshall007]

@pubkey are you saying we'll need the RethinkDB wrapper to emulate all of the "default strategies" you've specified there? RethinkDB has no internal concept of document versioning. That's something we had to begin implementing in Horizon in order to handle conflict resolution, so we'll have to think about that.

All updates made to single documents are guarunteed to be atomic. You can also read, then update a single document atomically, which would allow for exposing operations like the ones I've suggested in https://github.com/rethinkdb/horizon/issues/576.

[nevf]

@ewnd9

What do you think about pouchdb-find (which is used in this project) and find from CouchDB 2.0?

I haven't followed Couch/PouchDB development for quite a while now and the new query capabilities are very welcome .

I still have a problem with Couch/PouchDB/rxdb in my particular use case, in that there can be frequent small updates to moderately large documents, which will potentially dramatically increase the database size. I assume this may also affect performance. More of a concern is the increase in db size on mobile devices which are constrained by memory.

[pubkey]

After having this in mind for several weeks now, I will close this. The solution to combine RxDB with a firebase/rethinkDB-backend, lies not in changing RxDB to be compatible. What we need is a pouchdb-adapter for firebase/rethinkdb. This would not only give something back to the pouchdb-ecosystem, but also create a final solution which everyone can rely on.

[zoomclub]

How do we go about making such an adapter? Is there a PouchDB suggestion or issue board for this, how does the adapter get put forward and stay in the limelight?

[rdworth]

@zoomclub https://github.com/pouchdb/pouchdb/issues