Open togran opened 4 years ago
We've just added the Polkadot CC1 configuration files that reflects the settings on Polkascan.io, which I'm running at the moment for a testrun. You could pull the changes from the repo and test it out.
It could also be the case you need to force an type-registry update, the easiest to do this is to rebuild the Docker images by: docker-compose -p polkadot-cc1 -f docker-compose.polkadot-cc1-full.yml build --no-cache
Unfortunately, pulling fresh version and rebuilding containers doesn't solve the issue
Here's what happens (tried it a few times today to the same result):
rebuild --no-cache
and volume cleaning, starting from scratch using your docker-compose files with no modificationsBlock #3457 is missing.. stopping check
), block number is random from run to run.File "/usr/local/lib/python3.6/site-packages/scalecodec/types.py", line 1045, in process
raise ValueError("Index '{}' not present in Enum value list".format(self.index))
ValueError: Index '202' not present in Enum value list
Block #146002 is missing.. stopping check
Hope that helps to identify the problem because I'm at loss for now.
My best guess is that https://github.com/polkascan/polkascan-pre-harvester/blob/40003e2d3d2c5ca45b3e015e097851a53590da63/app/processors/converters.py#L811 remove_block happens, but
https://github.com/polkascan/polkascan-pre-harvester/blob/40003e2d3d2c5ca45b3e015e097851a53590da63/app/processors/converters.py#L815 add block doesn't for some reason
Blocks that can't be added by integrity checks are processed fine when calling harvester api's process endpoint later.
I suspect there are indeed some issues with processing the reorgs, as a workaround I modified the docker-compose configs in master with FINALIZATION_ONLY=1, so only finalized blocks will be processed.
This is actually the setting we also use at polkascan.io, so I'm pretty confident this should work, but let me know if these issues remain..
I'm pretty sure this will fix the problem, reorgs were obviously the culprits. Thanks!
With FINALIZATION_ONLY=1
it works fine, but I think the issue should stay open because without that option polkascan still fails to sync
Is this still the case with the latest version? Something you could also try is to monitor http://127.0.0.1:8080/polkadot-cc1/harvester/admin if blocks are being processed and try to restart by pressing "Process blocks in harvester queue" if process is interupted.
Occasionally harvester_worker raises an error and refuses to process blocks anymore
Might be related to Block #125428 but doesn't seem likely - it's always different blocks when restarted.
Also, not sure if related or not, aggregate tables (data_block_total, data_account etc) refuse to be filled, and it reflects on the GUI