RangeError: Could not decode varint

aalku commented 1 month ago

Hello!! Thanks for your project.

I'm getting this error just for being subscribed to the firehose, as you can see the error is inside @atproto/repo module but I don't know if that depends on your code or not. Or maybe it even depends on my code, but I think it doesnt.

I'm just subscribed and this happens, and as it's not in my code I don't know how to catch it and I would need to restart node.

Can you give any help? Thanks!!!

node:events:508
    const err = new ERR_UNHANDLED_ERROR(stringifiedEr);
                ^

Error [ERR_UNHANDLED_ERROR]: Unhandled error. ({
  cursor: '',
  error: RangeError: Could not decode varint
      at Object.read3 [as decode] (/project/node_modules/@skyware/firehose/node_modules/@atproto/repo/dist/index.js:13360:17)
      at readVarint (/project/node_modules/@skyware/firehose/node_modules/@atproto/repo/dist/index.js:25090:36)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async readHeader (/project/node_modules/@skyware/firehose/node_modules/@atproto/repo/dist/index.js:25095:19)
      at async decodeIterator (/project/node_modules/@skyware/firehose/node_modules/@atproto/repo/dist/index.js:25408:40)
})
    at Firehose.emit (node:events:508:17)
    at WebSocket.<anonymous> (file:/project/node_modules/@skyware/firehose/dist/index.js:64:22)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ERR_UNHANDLED_ERROR',
  context: {
    cursor: '',
    error: RangeError: Could not decode varint
        at Object.read3 [as decode] (/project/node_modules/@skyware/firehose/node_modules/@atproto/repo/dist/index.js:13360:17)
        at readVarint (/project/node_modules/@skyware/firehose/node_modules/@atproto/repo/dist/index.js:25090:36)
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
        at async readHeader (/project/node_modules/@skyware/firehose/node_modules/@atproto/repo/dist/index.js:25095:19)
        at async decodeIterator (/project/node_modules/@skyware/firehose/node_modules/@atproto/repo/dist/index.js:25408:40)
  }
}

stsysd commented 1 month ago

I was able to catch this exception by registering a listener for the error event on the Firehose.

const firehose = new Firehose();
firehose.on("error", (e) => console.error(e));

According to Node's EventEmitter specification, if there is no listener registered for the error event, the event is thrown as an exception.

I hope this helps!

aalku commented 1 month ago

It does. Thenks! Also I want to say this here: In this other issue they told me that those errors come from another federated network. I think they are invalid messages or something, so we have to deal with possible invalid records in the network, I don't think there is a way to avoid it completely.

Maybe this tool should handle those errors differently?

https://github.com/bluesky-social/atproto/issues/2484#issuecomment-2108997630

skyware-js / firehose

RangeError: Could not decode varint #1