search

streamingfast / firehose-ethereum

Ethereum on StreamingFast
Apache License 2.0
78 stars 24 forks source link

merger should delete old files upon detection #11

Closed matthewdarwin closed 1 year ago

matthewdarwin commented 2 years ago

Merger keeps finding new files that are old, but it doesn't delete them for some reason.

Re: https://github.com/streamingfast/sf-ethereum/issues/10

It seems to wait until it can merge a block, then delete the old files. But it might never merge a block because there are too many old files.

maoueh commented 2 years ago

@billettc @froch More work for you guys?

matthewdarwin commented 2 years ago

The test case is here is to have an mindreader running generating old one-block files while other mindreaders are generating latest one-block files.

matthewdarwin commented 2 years ago

image

Latest merger is really bad compared to the old merger. I've now stopped generating old blocks to make it stable. Let's see how it is over the next little bit.

matthewdarwin commented 2 years ago

BTW, count of one-block files went down because I have been manually removing them.

maoueh commented 1 year ago

I think it's fixed now.