RangerMauve / hyperswarm-web

Implementation of the hyperswarm API for use in web browsers
MIT License
102 stars 15 forks source link

Connection dies with larger writes #26

Closed jensenhertz closed 2 years ago

jensenhertz commented 2 years ago

I noticed a curious behavior. If you write much more than ~204800 bytes, the connection dies instantly. Is this expected behavior?

I don't have much in-depth experience with WebRTC, so excuse me if this is well-known thing or something ;)

Here's my test code...

const Swarm = require('../')

;(async () => {
  const hostname = 'your hyperswarm server instance'  
  const swarm = Swarm({
    bootstrap: [hostname],
  })

  const topic = Buffer.from('aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f', 'hex')

  swarm.once('connection', (connection, info) => {
    console.log('new connection!', info)
    connection.once('end', () => {
      console.log('Connection client1 -> client2 ended')
    })
    connection.once('data', (d) => {
      console.log('The client1 got data', d)
    })
    const b = Buffer.alloc(200*1024)
    console.log(`Sending ${b.byteLength} bytes`)
    connection.write(b)
  })

  swarm.on('disconnection', (socket, details) => {
    console.log(details.peer.host, 'disconnected!')
    console.log('now we have', swarm.peers.length, 'peers!')
  })

  swarm.join(topic)

  swarm.webrtc.signal.once('connected', () => {
    console.log('client webrtc connected')
  })

  await swarm.flush(()=>{})
})()
tinchoz49 commented 2 years ago

Hi @jensenhertz, the data channel of webrtc has a limited payload defined by the browser. You will need to use a split chunk solution to avoid this limitation.

https://tensorworks.com.au/blog/webrtc-stream-limits-investigation/#14-maximum-data-channel-message-size

jensenhertz commented 2 years ago

Hey @tinchoz49, I thought it might be something like that. It's going to be curious exercise to get hypercore replication to work with this.

tinchoz49 commented 2 years ago

Remember that hypercore also has a limitation of 8 mb by block.

What we did in https://sher.app is to persist hypercore blocks of 10 kb (maximum) by doing this we avoid the webrtc limitation since the hypercore.stream sends chunks of 10 kb

tinchoz49 commented 2 years ago

We also have a splitter module that split the packets before to send to webrtc and it does a resemble from the other side (peer)

tinchoz49 commented 2 years ago

I should publish that module :S I will try to do it next week

jensenhertz commented 2 years ago

@tinchoz49 I was just thinking that I need to write some sort of splitter, so if you have that already and you're willing to publish it, that would be perfect! 🙏

tinchoz49 commented 2 years ago

https://github.com/geut/splitter

jensenhertz commented 2 years ago

Thank you, that worked!