search

geut / sodium-javascript-plus

sodium-javascript with experimental support for xchacha20 and kx
MIT License
16 stars 0 forks source link

Cannot replicate with hyperdrive-daemon #1

Closed sammacbeth closed 4 years ago

sammacbeth commented 4 years ago

I've been trying to get replication from a hyperdrive10 in the browser with hyperdrive-daemon working and ran into issues. Replication breaks down at the handshaking step of the hypercore protocol. This can be reproduced on the experimental dat2 branch of the dat-sdk (https://github.com/datproject/sdk/tree/dat2-take-two).

I added a test case on my fork that triggers this case. If we catch the replication error in this case we get:

Error: "Decoded message is not valid"
    decode http://127.0.0.1:8080/test-bundle.js:66646
    _onhandshake http://127.0.0.1:8080/test-bundle.js:66216
    finish http://127.0.0.1:8080/test-bundle.js:66436
    run http://127.0.0.1:8080/test-bundle.js:58453
    drainQueue http://127.0.0.1:8080/test-bundle.js:58417

This comes from simple-hypercore-protocol when decoding the remote payload during handshaking. If we inspect the remotePayload that is being decoded it is a Uint8Array of length 26 with all zeros. This indicates that something goes wrong before this point, either causing the hyperdrive-daemon to send a bad response to the handshake, or we failed to decode their response.

Screenshot 2020-04-15 at 18 13 34

I'm now trying to get a simpler reproduction case. I tweaked the hypercore8 browser example to test sync between node and the browser: https://github.com/sammacbeth/workaround-hypercore8-browser/commit/d08766c1d3c0c60819ca8105915b5e725c896809 This test case as a server node server.js, then the browser test case (npm run start) will try to replicate a hypercore with it over a websocket. This works, so it seems that the issue lies somewhere else in the stack. I'll try to build up this test case to find it and update this thread with my progress.

tinchoz49 commented 4 years ago

Hey @sammacbeth can you try the latest version?

tinchoz49 commented 4 years ago

I try this https://github.com/sammacbeth/workaround-hypercore8-browser/commit/d08766c1d3c0c60819ca8105915b5e725c896809 with the v0.0.5 and it's working. I had an issue related with the C stack size for the wasm libraries, now it should be compatible with the sodium-native operations.

sammacbeth commented 4 years ago

Hi @tinchoz49. Sorry if it wasn't clear - the test case in workaround-hypercore8-browser did not reproduce the issue. Currently I can only reproduce in dat-sdk and in my own code. I tested the latest version of this repo and the issue persists.

tinchoz49 commented 4 years ago

Hey Sam, if you run workaround-hypercore8-browser with the broked version of sodium-javascript-plus (that means v0.0.4 not v0.0.2) you will see this error: 

Error: Could not parse remote payload
    at SimpleProtocol._onhandshake (index.js:151)
    at finish (handshake.js:64)
    at Item.run (browser.js:153)
    at drainQueue (browser.js:123)

If you update to v0.0.5 the error is gone and start replicating.

I also did a test with a clean simple-hypercore-protocol where I got to reproduce the error too and now it's working, so simple-hypercore-protocol between sodium-native and sodium-javascript-plus should be compatible now.

sammacbeth commented 4 years ago

Ah looks like I was mistaken, I just went through the tests again and this seems to be working! Great job!

tinchoz49 commented 4 years ago

That's great to hear! I put the example here just in case: https://github.com/tinchoz49/test-simple-hypercore-protocol