search

streamingfast / firehose-core

Firehose Integrators Tool Kit (for `firehose-<chain>` maintainers)
Apache License 2.0
7 stars 9 forks source link

Add `--lib` command line argument to RPC poller #52

Open matthewdarwin opened 1 month ago

matthewdarwin commented 1 month ago

If you are processing old block ranges with the RPC poller + merger and want to chunk up the block range into separate jobs, you need to overlap the different jobs by sufficient blocks to cover the lib. Else the last set of one-blocks will not be merged.

Add option --lib 0 or something to force the lib to be 0 instead of the default 200.

Or maybe just set the lib to 1 block automatically if the head_block_drift > 6 hrs.

Current one-block filenames show 200 block LIB, example:

0032000025-9a76f217563354bb-621c32590ba799b2-31999825-default.dbin.zst
matthewdarwin commented 4 weeks ago

I suppose FORCE_FINALITY_AFTER_BLOCKS = 1 is workaround for this?

matthewdarwin commented 4 weeks ago

Actually probably not a good idea.
Rather want a way to force the merger to merge without waiting for lib.