mafintosh / hyperdb

Distributed scalable database
MIT License
753 stars 75 forks source link

createHistoryStream (live) omitting entries #40

Closed andrewosh closed 6 years ago

andrewosh commented 6 years ago

Hey all,

I'm having trouble with the following createHistoryStream example:

const hyperdb = require('hyperdb')
const ram = require('random-access-memory')

let db1 = hyperdb(ram)
db1.ready(() => {
  let db2 = hyperdb(ram, db1.key)
  db2.ready(() => {
    db1.authorize(db2.key, () => {
      let rep1 = db1.replicate({ live: true })
      let rep2 = db2.replicate({ live: true })
      rep2.pipe(rep1).pipe(rep2)

      let historyStream = db1.createHistoryStream({ live: true })
      historyStream.on('data', console.log)
      db1.put('/hey', 'there')
      db2.put('/yo', 'there')
      db1.put('/dog', 'dog')
      db2.put('/thing', 'thing')

      setTimeout(() => {
        db2.get('/hey', (_, contents) => {
          console.log('got contents:', contents)
        })
      }, 1000)
    })
  })
})

Even though db2 is an authorized writer, historyStream only emits the changes to the /hey and /dog keys (both changed on db1). When I register the history stream on db2, I see all four changes. Similarly, db2 gets the value of /hey after that delay.

Anything I'm clearly doing wrong here?

andrewosh commented 6 years ago

Should add that this is on the latest master

Mayeu commented 6 years ago

Could it be related to recently merged PR?

hackergrrl commented 6 years ago

Tiny thing in your code that's causing the problem: the line that authorizes the 2nd hyperdb with the 1st is

    db1.authorize(db2.key, () => {

but db.key always refers to the shared key. So what you're doing here is having db1 re-authorize itself. You want the db's local key, which is db2.local.key, not db2.key. If you make this change it should replicate & iterate history cleanly.