Test Failure: disaster_recovery_2

linh2931 commented 2 months ago

https://github.com/AntelopeIO/spring/actions/runs/10048012787/job/27771514855?pr=382#step:4:1595

Traceback (most recent call last):
  File "/__w/spring/spring/build/tests/disaster_recovery_2.py", line 140, in <module>
    assert node.waitForLibToAdvance(), "Node did not advance LIB after relaunch"

heifner commented 1 month ago

The test restarts 5 nodes from a snapshot. The test failed because node0 could not connect to any node when launched so it waited 30 seconds before trying again. During this 30 seconds it happened to be producing blocks. Once it was able to connect it was busy feeding blocks to the other node.

Options:

Modify the test to wait longer than the default 30 seconds for LIB to advance. Note the default for reconnect attempts is 30 seconds.
Modify net_plugin to attempt connections quicker than 30 seconds at launch and back off to configured connection monitor time after a number of connection attempts.

spoonincode commented 1 month ago

appears to still exist? https://github.com/AntelopeIO/spring/actions/runs/10272637950/job/28425737014 reopening for now but if felt strongly can open new one instead

spoonincode commented 1 month ago

noticed again on https://github.com/AntelopeIO/spring/actions/runs/10376337986/job/28728623008 though on an UBSAN build which of course goes kinda on the slow side. Should we reopen this one or do a new issue?

AntelopeIO / spring

Test Failure: disaster_recovery_2 #390