MystenLabs / mysticeti

Mysticeti: Low-Latency DAG Consensus with Fast Commit Path
Apache License 2.0
55 stars 28 forks source link

fix shutdown process of validator #43

Closed akichidis closed 1 year ago

akichidis commented 1 year ago

After wiring Mysticeti to SUI it looks like it wasn't able to properly shutdown giving us errors when trying to bootstrap again the validator node like:

Failed to bind to local socket: Os { code: 48, kind: AddrInUse, message: "Address already in use" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

which basically raised suspicions of networking not properly shutting down and potentially a few other components too.

By using a simple test that starts and shutdown a validator not and using tokio-console we could see that some tasks were not terminating (amongst them network as well occupying the current port)

Screenshot 2023-11-04 at 01 31 03

after introducing explicit handling on propagating shutdown signals to the task components after shutting down the validator we can see that none of the tasks is running any more:

Screenshot 2023-11-04 at 01 25 06

a test has been introduced for this case which is successfully passing after the introduced changes. Running the same test in main fails with a Failed to bind to local socket: Os { code: 48, kind: AddrInUse, message: "Address already in use" } error.