trace-history/chain-state-history option of state_history_plugin breaking with v1.0.1's snapshot-import/node running

[jun0tpyrc]

did like 30 times using different snapshot sources to confirm this

Both of these flags are not safe to add in import/migrate process

trace-history = true 
chain-state-history = true   

not ok to add back afterwards as well we using them as suggested for hyperion indexer setup

What's worse is as long as chain-state-history = true is involved , it will not print error detais but only the appbase quit called result ...

Otherwise( just using trace-history = true ) , could see

wrote != -1: write failure on file /usr/opt/eosio/data/state-history/trace_history.log: Operation not supported
    {"fn":"/usr/opt/eosio/data/state-history/trace_history.log","e":"Operation not supported"}
    nodeos  random_access_file.hpp:136 write_to

info  2024-09-13T03:56:48.831 nodeos    main.cpp:172                  operator()           ] appbase quit called
warn  2024-09-13T03:56:48.831 nodeos    controller.hpp:493            emit                 ] /__w/spring/spring/libraries/chain/controller.cpp:3419 controller_emit_signal_exception: 3140002 state_history_write_exception: State history write error
State history encountered an Error which it cannot recover from.  Please resolve the error and relaunch the process
    {}
    nodeos  state_history_plugin.cpp:153 on_accepted_block

error 2024-09-13T03:56:48.831 nodeos    controller.cpp:3809           apply_block          ] e.to_detail_string(): 3140002 state_history_write_exception: State history write error
State history encountered an Error which it cannot recover from.  Please resolve the error and relaunch the process
    {}
    nodeos  state_history_plugin.cpp:153 on_accepted_block

error 2024-09-13T03:56:48.831 nodeos    controller.cpp:4473           operator()           ] exception thrown while switching forks 3140002 state_history_write_exception: State history write error
State history encountered an Error which it cannot recover from.  Please resolve the error and relaunch the process
    {}
    nodeos  state_history_plugin.cpp:153 on_accepted_block

    {}
    nodeos  controller.cpp:3817 apply_block

[jun0tpyrc]

tested with 1.0.0/1.0.1

[jun0tpyrc]

mainnet only , jungle4 testnet is fine, verified with 1) https://snapshots.eosnation.io/ <= v6 eos-mainnet snapshot 2) https://store3.eosphere.io/eos/snapshots/ also 3) leapv4 private snapshot taken by self

[spoonincode]

Operation not supported

Are you running on Ubuntu 18? Leap 4.0 was the last version to support Ubuntu 18.

If you want to continue using Ubuntu 18 with state_history_plugin enabled, you will need to install the HWE kernel. This is fully supported by Ubuntu and easy to do, https://ubuntu.com/kernel/lifecycle#installation-18-04 However be aware we do not run any tests on Ubuntu 18 so it is at your own risk to run in such an environment.

[jun0tpyrc]

Operation not supported

Are you running on Ubuntu 18? Leap 4.0 was the last version to support Ubuntu 18.

If you want to continue using Ubuntu 18 with state_history_plugin enabled, you will need to install the HWE kernel. This is fully supported by Ubuntu and easy to do, https://ubuntu.com/kernel/lifecycle#installation-18-04 However be aware we do not run any tests on Ubuntu 18 so it is at your own risk to run in such an environment.

ubuntu-22.04 with this repo's prebuilt package , or you could try reproduce this issue with dockerize

FROM ubuntu:22.04
ARG RELEASE_TAG

WORKDIR /opt/app/
RUN apt-get update && apt-get install wget libcurl4 libusb-1.0-0 -y
RUN wget https://github.com/AntelopeIO/spring/releases/download/v${RELEASE_TAG}/antelope-spring_${RELEASE_TAG}_amd64.deb -O spring.deb
RUN dpkg -i spring.deb
ENTRYPOINT ["nodeos"]

VERSION=1.0.1
docker build . -t spring:v${VERSION} --build-arg RELEASE_TAG=${VERSION} --no-cache

leaving state_history_plugin on without those 2 options would run fine though , but wonder if some functions missing

[spoonincode]

What's the host distro that is running the container? More specifically the requirement is a Linux kernel running 4.16+ (this is over 6 years old)

[jun0tpyrc]

seem this could be related then

Linux a362994b750b 4.14.350-266.564.amzn2.x86_64 #1 SMP Sat Aug 10 09:56:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Could this runtime-dependency be clarified / have clearer error message ?! Thought only the kernel/OS version of container image matter but did not expect some features need pass through from host's kernel

v verified below

[jun0tpyrc]

Thanks, verified pumping to

5.15.165-110.161.amzn2.x86_64 #1 SMP Fri Aug 23 18:41:15 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 

would work

One would hit this if still running amznlinux2 without custom kernel series pump ...

[arhag]

Currently, we only officially support the following OSes:

• Ubuntu 22.04 Jammy
• Ubuntu 20.04 Focal

And while the README does not currently make this explicit, we also only officially support running Spring on versions of the Linux kernel that officially come with these versions of the Ubuntu distro. That distinction is important in cases like this one where one is running Spring software in a containerized supported OS running on a host OS that is not supported by Spring.

So while Ubuntu 22.04 is supported, the combination of the kernel and OS versions you used to reproduce the undesired behavior is not supported by Spring.

The best solution is the one you already got to which is to update the kernel to a supported version.

That all said, as we mention in the README:

Other Unix derivatives such as macOS are tended to on a best-effort basis and may not be full featured.

As part of that "best-effort" basis, we have in this particular case decided to provide a fallback in the case of old kernels that should resolve this particular issue.