search

cculianu / Fulcrum

A fast & nimble SPV Server for BCH, BTC, and LTC
Other
325 stars 73 forks source link

Add FulcrumAdmin to the docker container to be able to gracefully shutdown Fulcrum #253

Closed antonr-p2p closed 1 month ago

antonr-p2p commented 1 month ago

Please add FulcrumAdmin to the docker container to be able to gracefully shutdown Fulcrum. Faced issue with corrupted DB, because used kubectl delete pod fulcrum-0 today. Many times before it survived, but today faced corrupted data dir. Once you would add FulcrumAdmin it would be possible to configure shutdown hook.

cculianu commented 1 month ago

Fulcrum responds to graceful shutdown signals that are standard such as SIGTERM and SIGINT.

just don’t kill it with SIGKILL.

Also provide a timeout to wait for graceful shutdown.

cculianu commented 1 month ago

I'll add the FulcrumAdmin script to the built image. The fact that it's missing is indeed an error. I think python3 also needs to be installed for it to work but that shouldn't be a problem.

cculianu commented 1 month ago

Ok, added to image. I will do a release very soon (v. 1.11.0 -- good stuff coming!). I will see if I can do a github action now to publish the new image tho.

antonr-p2p commented 1 month ago

@cculianu thanks a lot for a quick response and actions! In a meantime I would try to use terminationGracePeriodSeconds option, I hope it would help.

cculianu commented 1 month ago

You're welcome. Yeah I'm no kubernetes/docker expert but it should be possible to gracefully shut down the container rather than nuke it.. like sending all processes a SIGTERM seems customary. Pretty sure if you dig a bit as to how to wait for graceful shutdown, it should work ok.

But yeah I just pushed a new image and FulcrumAdmin now lives alongside Fulcrum now in /usr/bin inside the image.

antonr-p2p commented 1 month ago

BTW, Kubernetes send SIGTERM and waits 30 seconds by default, then only SIGKILL. This delay can be configured via terminationGracePeriodSeconds . My issue was that I provided too small memory for both DB and container itself, so it crashed and had DB corrupted. Once I gave enough memory - I had no issue so far.