Better error handling for "missing state folder protects statehist logs from being purged"

[eosusa]

Apparently in the latest releases (im on 3.2.4), the feature that attempts to protect a node from accidentally wiping out all the existing state history files by kicking off a --replay-blockchain (https://github.com/AntelopeIO/leap/issues/659) is working (i had removed by state directory and had existing statehist logs) but the error message returned did not really lead me to the conclusion i needed to either pass --delete-state-history along with the --replay-blockchain (or manually purge them myself).

warn  2023-07-13T13:41:55.910 nodeos    controller.cpp:457            initialize_blockchai ] Initializing new blockchain with genesis state
info  2023-07-13T13:41:55.913 nodeos    controller.cpp:495            replay               ] existing block log, attempting to replay from 2 to 1428000 blocks
warn  2023-07-13T13:41:55.914 nodeos    controller.cpp:369            emit                 ] controller_emit_signal_exception: 3140002 state_history_write_exception: State history write error
State history encountered an Error which it cannot recover from.  Please resolve the error and relaunch the process
    {}
    nodeos  state_history_plugin.cpp:519 on_accepted_block

error 2023-07-13T13:41:55.914 nodeos    controller.cpp:2129           apply_block          ] e.to_detail_string(): 3140002 state_history_write_exception: State history write error

Being this wasn't the way previous versions worked, I had no clue the real issue was this new feature requiring the statehist logs to be cleared out, so maybe indicating that in the error message (vs a generic "cant write" error) would be more intuative for the operator to know how to resolve (pass the --delete-state-history param)

[bhazzard]

@eosusa Can you confirm whether you are loading a snapshot with a block earlier than the blocks in your state history log?

@heifner can you confirm that 4.0.0 handles this error message in a better way?

[eosusa]

this wasnt a snapshot load, this was a --replay-blockchain that was requested on a node i had to manually purge out the state directory on due to another issue. when i relayed it over to kevin (because the error is an ambiguous cannot write error), he said that was due to this new feature thats trying to protect those statehist logs from accidental deletion. i can confirm that passing the --delete-state-history param along with the --replay-blockchain param worked, i just had no clue that was the recommended path because the error just said generic statehist write error (edit: and this --delete-state-history param wasnt required on earlier versions so didn't know that was even an option)

[heifner]

In 4.0: rm -rf d/state ./nodeos --data-dir d --config-dir d --plugin eosio::state_history_plugin --disable-replay-opts --trace-history -e -p eosio --replay-blockchain

Produces error:

warn  2023-07-26T20:51:09.655 nodeos    controller.cpp:485            initialize_blockchai ] Initializing new blockchain with genesis state
info  2023-07-26T20:51:09.675 nodeos    controller.cpp:523            replay               ] existing block log, attempting to replay from 2 to 20 blocks
info  2023-07-26T20:51:10.098 nodeos    platform_timer_accurac:62     compute_and_print_ti ] Checktime timer accuracy: min:1us max:1755us mean:9us stddev:25us
error 2023-07-26T20:51:10.101 nodeos    state_history_plugin.c:302    on_accepted_block    ] fc::exception: 3110000 plugin_exception: Plugin exception
Existing ship log with 22 blocks when starting from genesis block 2
    {"eb":22,"b":2}
    nodeos  log.hpp:513 write_entry

warn  2023-07-26T20:51:10.104 nodeos    controller.cpp:392            emit                 ] controller_emit_signal_exception: 3140002 state_history_write_exception: State history write error
State history encountered an Error which it cannot recover from.  Please resolve the error and relaunch the process
    {}
    nodeos  state_history_plugin.cpp:310 on_accepted_block

error 2023-07-26T20:51:10.104 nodeos    controller.cpp:2171           apply_block          ] e.to_detail_string(): 3140002 state_history_write_exception: State history write error
State history encountered an Error which it cannot recover from.  Please resolve the error and relaunch the process
    {}
    nodeos  state_history_plugin.cpp:310 on_accepted_block

warn  2023-07-26T20:51:10.105 nodeos    controller.cpp:2331           replay_push_block    ] 3140002 state_history_write_exception: State history write error
State history encountered an Error which it cannot recover from.  Please resolve the error and relaunch the process
    {}
    nodeos  state_history_plugin.cpp:310 on_accepted_block

    {}
    nodeos  controller.cpp:2179 apply_block

error 2023-07-26T20:51:10.111 nodeos    main.cpp:161                  main                 ] 3140002 state_history_write_exception: State history write error
State history encountered an Error which it cannot recover from.  Please resolve the error and relaunch the process
    {}
    nodeos  state_history_plugin.cpp:310 on_accepted_block

    {}
    nodeos  controller.cpp:2179 apply_block
rethrow
    {}
    nodeos  controller.cpp:2331 replay_push_block

    {}
    nodeos  chain_plugin.cpp:1139 plugin_startup