calling `archive.finalize()` makes the stream returned from `createFileReadStream` not receive any data

[poga]

I've encountered a weird issue, not sure if it's intended.

I have two file: index.js

var hyperdrive = require('hyperdrive')
var level = require('level')
var swarm = require('hyperdrive-archive-swarm')

var db = level('./test.db')
var drive = hyperdrive(db)
var archive = drive.createArchive()

var ws = archive.createFileWriteStream('foo')
ws.write('hello')
ws.write('world')
ws.end()

var link = archive.key.toString('hex')
console.log(link, '<-- this is your hyperdrive link')
archive.finalize(() => {
  archive.list((err, entries) => { console.log(err, entries) })
})

var sw = swarm(archive)

peer.js

var swarm = require('hyperdrive-archive-swarm')
var hyperdrive = require('hyperdrive')
var level = require('level')

var db = level('./another-hyperdrive.db')
var drive = hyperdrive(db)

var link = new Buffer('<link from index.js>', 'hex')
var archive = drive.createArchive(link)
var sw = swarm(archive)

archive.finalize((err) => {
  var rs = archive.createFileReadStream('foo')
  // rs won't receive any data
  rs.on('data', x => {console.log(x)})
  rs.on('error', x => { console.log(err) })
})

As in the example, if I call archive.finalize() on an archive I'm not owning it. Any file read stream will not receive any data. no event listener is called. If I remove the archive.finalize() in peer.js, the stream works fine.

I think if I shouldn't call archive.finalize() in the situation. It should raise or return an error.

More question: When should I use archive.finalize()? Should I only call it on a non-live & owned archive? What's the purpose of it?

[joehand]

When should I use archive.finalize()? Should I only call it on a non-live & owned archive? What's the purpose of it?

Ya, you should only call archive.finalize() on owned archives. Finalizing requires write permission on the archive. It will write the metadata allowing you to share.

If you remove archive.finalize, the stream should receive data.

I think if I shouldn't call archive.finalize() in the situation. It should raise or return an error.

Definitely. I ran this with memdb first instead of level and it did throw a weird error. It may be calling back with an error, but mine wasn't.

We definitely callback with an error if you run finalize with unowned archives.

[poga]

Thanks for the clarification.

I tried memdb and got this error:

/Users/poga/projects/hd_test/node_modules/sodium-signatures/sodium.js:9
  return sodium.crypto_sign_verify_detached(signature, message, publicKey)
                ^

Error: argument publicKey must be a buffer
    at Error (native)
    at Object.exports.verify (/Users/poga/projects/hd_test/node_modules/sodium-signatures/sodium.js:9:17)
    at loop (/Users/poga/projects/hd_test/node_modules/hypercore/lib/feed.js:408:23)
    at Feed._putRoots (/Users/poga/projects/hd_test/node_modules/hypercore/lib/feed.js:384:3)
    at loop (/Users/poga/projects/hd_test/node_modules/hypercore/lib/feed.js:268:10)
    at Feed.put (/Users/poga/projects/hd_test/node_modules/hypercore/lib/feed.js:229:3)
    at write (/Users/poga/projects/hd_test/node_modules/hypercore/lib/replicate.js:254:13)
    at Channel.ondata (/Users/poga/projects/hd_test/node_modules/hypercore/lib/replicate.js:175:43)
    at emitOne (events.js:96:13)
    at Channel.emit (events.js:188:7)

However, I didn't receive any error in the callback with level. peer.js

var swarm = require('hyperdrive-archive-swarm')
var hyperdrive = require('hyperdrive')
var level = require('level')

var db = level('./another-hyperdrive.db')
var drive = hyperdrive(db)

var link = new Buffer('409151b9ac9a00bf1ad01ab9403c93b3b74ace501dec7b4b33f7b4a14144c0ed', 'hex')
var archive = drive.createArchive(link)
var sw = swarm(archive)

archive.finalize((err) => {
  // err === null
  var rs = archive.createFileReadStream('foo')
  rs.on('data', x => {console.log(x)})
  rs.on('error', x => { console.log(err) })
})

version: hyperdrive: 7.6.0 hyperdrive-archive-swarm: 4.1.6 level: 1.4.0

[mafintosh]

@poga which version of node to you use? that last error looks weird indeed

[poga]

@mafintosh I tried with 6.5.0 and 6.6.0. both have the same result.

[poga]

I tried to find the root cause of the issue.

Looks like the issue is in hypercore. I encountered some TypeError in hypercore's test so I'm current stuck there.

Just opened https://github.com/mafintosh/hyperdrive/pull/101 . Maybe add some notes about archive() will help some other people.