search

ssbc / ssb-db

A database of unforgeable append-only feeds, optimized for efficient replication for peer to peer protocols
https://scuttlebot.io/
MIT License
1.17k stars 75 forks source link

Bootstrap PouchDB API #189

Closed drschwabe closed 6 years ago

drschwabe commented 6 years ago

Is it a stretch to make ssb compatible with PouchDB/CouchDB replication ?

PouchDB has an amazing database API and it would be equally as amazing to sort of 'swap' the backend to a ssb.

Better yet, the ability to mount a ssb into a new PouchDB would be insanely cool. Ie- 

var ssb = require('secure-scuttlebutt/create')(pathToDB)
var PouchDB = require('pouchdb')
var mySSBPouchDB = new PouchDB(ssb)

Or vice versa: 

var PouchDB = require('pouchdb')
var myPouchDB = new PouchDB(pathToDB)
var mySSBPouchDB = require('secure-scuttlebutt/create')(myPouchDB)

At this point you could use it like any PouchDB; each document now it's own "feed" with the messages stored as fields in said document. Ie- do a one time PouchDB POST call to initiate it ie- mySSBPouchDB.post({ "cool" : true, { "awesomeness" : 9001}) it will make the _id a unique public key and perhaps return you a private key for which you can use to update the doc again ie- via PouchDB PUTs. Extra points if we can also use PouchDB's replication API (with the results automagically 'secure-scuttlebuttified').

I recently opened a thread on SX floating the idea of making a 'hashchain' based revision algorithm (plugin) for PouchDB's - but now am thinking the strategy above would perhaps be a better way to go about achieving the same goal.

As such, any opinion on the do-able'ness of this or perhaps insight on improved technique/architecture would be appreciated!

dominictarr commented 6 years ago

Sometimes being able to abstract over databases and switch out the internal implementation seems like a good idea, but this restricts you to the common set of interfaces that the underlying databases all support. That means you can't use any of the strengths of a particular database design - ssb and couch both try to do particular things - the shared subset of both databases wouldn't be very interesting. I think it's better to let couch be couch and ssb be ssb. If you want a signed chain security oriented database - then you want ssb! I built ssb because I wanted this and because this wasn't what couch was.

The ssb design was influenced by couchdb - but couch is more suited for a trusted enviroment, it has replication but it's not fully peer to peer. ssb makes certain compromises to optimize for replication, really it goes further than couch, will still tries to look somewhat like an ordinary database.

the two main differences:

  1. couchdb is a key:value store with mutable documents. You can update a document, and rules about whether that is a valid write is run in a javascript function, (on the server, in practice). Instead up straight up mutable documents, ssb only has messages. It's like if the state of every couch document was generated as a map-reduce view. This makes replication work better, because it never has to retrive a different revision for a document (or has a conflict!). You can still represent mutable documents in ssb, by having multiple messages that "reply" to each other.

  2. couchdb has a sequence number for updates locally received on each peer. This works best with more star-shaped network topologies (i.e. single server, or few servers), if you have more than one server, different updates will end up with different sequence numbers on each server... long story short, this means lots of http requests. ssb instead has a sequence number per user - you just say what sequence number you where up to and the peer sends you anything you didn't have - one round trip only! Also, because each message is signed and every feed is append-only it works in a fully random network, and you also don't have to replicate the same feeds as another peer.

drschwabe commented 6 years ago

Man that is an excellent point. Certainly best then to use the strengths of each and not limit to a common subset of features that overlap simply for the goal of mixmashing APIs. I think the underlying issue here is that I'm just being lazy about learning something new :)

Thanks for clarifying the main differences in detail - this is super helpful. I like what you are doing here and need a working system in production soon so will stop being lazy and start hacking!

dominictarr commented 6 years ago

If you have questions about how ssb replication works, the best place to ask is on ssb ;) easiest client to install is: https://github.com/ssbc/patchwork and get a pub invite from https://github.com/ssbc/scuttlebot/wiki/Pub-Servers