search

streamingfast / firehose-core

Firehose Integrators Tool Kit (for `firehose-<chain>` maintainers)
Apache License 2.0
8 stars 10 forks source link

firehose exits before zipping all the one block files #53

Open matthewdarwin opened 1 month ago

matthewdarwin commented 1 month ago

If I start the RPC poller from firehose and give it a defined range of blocks to create, it creates all the "dbin" files, but some files don't get a "dbin.zst" extension.... The reader exits before all the zipping is completed, which leaves incomplete block range for the merger (running in a separate process).

This is noticeable if the server running the rpc poller is under heavy load.

/usr/bin/fireeth -d /tmp/data -c '' start reader-node --reader-node-path /usr/bin/fireeth --common-first-streamable-block 40100000  --reader-node-stop-block-num 40110001 --reader-node-arguments tools poller generic-evm http:///xxxxxxx 40100000 --data-dir=/tmp/data
matthewdarwin commented 1 month ago

Also the cursor needs to be updated AFTER the file is zipped.