ChainSafe / js-libp2p-yamux

Typescript implementation of Yamux
Other
12 stars 9 forks source link

`Error: data limit exceeded` when streaming large data through relay node #56

Closed noahfigueras closed 1 year ago

noahfigueras commented 1 year ago

I'm trying to migrate from mplex to yumex as it's the recommended by libp2p. But, with mplex I can transfer large sets of data with no problem, but every time I try with yumex the transfer loses chunks of data. I tried setting bigger limits and also transfering data through multiple streams but I lose data almost every time and my relay node in libp2p always gives me the following error Error: data limit exceeded. I wonder if there's any way I can push a long array of data through a stream ?

ERROR:

libp2p:circuit-relay:utils:error error while relaying streams dst -> src Error: data limit exceeded
    at countStreamBytes (file:///home/nf/Blockchain/smoothly_protocol/relay/node_modules/libp2p/dist/src/circuit-relay/utils.js:23:19)
    at countStreamBytes.next (<anonymous>)
    at duplex.sink (file:///home/nf/Blockchain/smoothly_protocol/relay/node_modules/it-pb-stream/dist/src/index.js:38:26)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) +55s
  libp2p:yamux:error stream abort id=81 error=Error: data limit exceeded
    at countStreamBytes (file:///home/nf/Blockchain/smoothly_protocol/relay/node_modules/libp2p/dist/src/circuit-relay/utils.js:23:19)
    at countStreamBytes.next (<anonymous>)
    at duplex.sink (file:///home/nf/Blockchain/smoothly_protocol/relay/node_modules/it-pb-stream/dist/src/index.js:38:26)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) +55s
  libp2p:yamux:error stream source ended id=81 CodeError: Error: data limit exceeded
    at YamuxStream.abort (file:///home/nf/Blockchain/smoothly_protocol/relay/node_modules/@chainsafe/libp2p-yamux/dist/src/stream.js:199:22)
    at abortStreams (file:///home/nf/Blockchain/smoothly_protocol/relay/node_modules/libp2p/dist/src/circuit-relay/utils.js:31:13)
    at file:///home/nf/Blockchain/smoothly_protocol/relay/node_modules/libp2p/dist/src/circuit-relay/utils.js:67:13
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ERR_STREAM_ABORT',
  props: {}
} +0ms
noahfigueras commented 1 year ago

I'm closing this, as I found out it is not related to the yamux package. I had to increase the defaultDataLimit on the libp2p relay node circuitRelayServer() . Following is the config that solved my issue:

    const relayNode = await createLibp2p({
        peerId: id,
        addresses: {
            listen: ['/ip4/0.0.0.0/tcp/37465/ws'],
            //announce: ['/dns4/127.0.0.1/tcp/443/wss/']
        },
        transports: [
            webSockets(),
            circuitRelayTransport()
        ],
        connectionEncryption: [
            noise()
        ],
        streamMuxers: [
            yamux({ 
                maxMessageSize: 1 << 20
            })
        ],
        services: {
            identify: identifyService(),
            relay: circuitRelayServer({
                reservations: {
                    defaultDataLimit: BigInt(1 << 20), // the default maximum number of bytes that can be transferred over a relayed connection
                }
            }),
            pubsub: floodsub() 
        },
        peerDiscovery: [
            pubsubPeerDiscovery({
                interval: 1000
            })  
        ],
    })