search

streamingfast / firehose-aptos

Firehose Aptos
Apache License 2.0
6 stars 3 forks source link

Connection to firehose broken #16

Closed ppoliani closed 1 year ago

ppoliani commented 1 year ago

I've been using a custom sink similar to this one https://github.com/streamingfast/aptos-indexer/. It's been working fine so far until I update firehose-aptos to the latest version.

There seems to be some issues with the TCP connections. More specifically I'm seeing the following error when I run the sink.

Received error An error occurred while streaming blocks: Status { code: Unknown, message: "error reading a body from connection: broken pipe", source: Some(hyper::Error(Body, Error { kind: Io(Kind(BrokenPipe)) })) }
Processor stopped
2023-02-02T08:15:04.544516Z ERROR lapin::io_loop: error doing IO error=IOError(Custom { kind: Other, error: "IO driver has terminated" })
2023-02-02T08:15:04.544603Z ERROR lapin::channels: Connection error error=IO error: IO driver has terminated
maoueh commented 1 year ago

Were you able to run and received some blocks or never succeed to connect and stream something with the new version?

ppoliani commented 1 year ago

Were you able to run and received some blocks or never succeed to connect and stream something with the new version?

it literally disconnects straightaway

maoueh commented 1 year ago

Ok that's a bit strange at first sight. Do you see something in firehose logs when doing the connection?

ppoliani commented 1 year ago

Ok that's a bit strange at first sight. Do you see something in firehose logs when doing the connection?

I will try to further debug this and let you know with what my findings are

ppoliani commented 1 year ago

@maoueh sorry for the late response. So I did finally give it a go this morning. I setup everything locally to check the logs. It turns out that the issue is still this one https://github.com/streamingfast/firehose-aptos/issues/15. I could easily see it on the remote server because it was running in a process manager so it'd restart the process. But now that I run this locally, I can see that the moment the Sink connect to firehose the same error appears. I'm running the latest commit #5d683d6

Let me share the share the error logs again. 

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x70 pc=0x103e7cfac]

goroutine 2210 [running]:
github.com/streamingfast/substreams/service.(*StreamFactory).New(0x1400014aa20, {0x105580660, 0x140002899e0}, {0x1055654a0, 0x140007f4000}, 0x103e7baf0?, 0x9184e72a000, {0x0, 0x0}, 0x0)
        /Users/xxxxx/go/pkg/mod/github.com/streamingfast/substreams@v0.2.1-0.20230130195638-895599b398e8/service/stream.go:47 +0x2ac
github.com/streamingfast/substreams/service.(*Service).blocks(0x1400079def0, {0x105580660, 0x140005bd080}, {0x3e8, 0x61a8, 0x2, {0x10558c1b8, 0x140004377c0}, 0x14000abbf00, 0x0}, ...)
ppoliani commented 1 year ago

@maoueh I have updated to the latest substream version 1.0.0 to no avail. I still see the same error which.

ppoliani commented 1 year ago

This PR fixes this issue https://github.com/streamingfast/firehose-aptos/pull/17/files