nats-io / nats-server

High-Performance server for NATS.io, the cloud and edge native messaging system.
https://nats.io
Apache License 2.0
16k stars 1.41k forks source link

NRG: Don't delete RAFT state if stream/consumer creation failed during shutdown #6061

Closed MauriceVanVeen closed 3 weeks ago

MauriceVanVeen commented 3 weeks ago

Antithesis found a case where if a server was hard killed and would replay RAFT append entries during startup, if it would then be SIGTERM-ed while the RAFT recovery is running, the acc.lookupStream that happens in js.processClusterCreateStream resulted in a JetStream not enabled for account error. Which would then delete the RAFT state.

There are also other errors that could pop up, so either way; if we're shutting down, don't delete the RAFT state.

Signed-off-by: Maurice van Veen github@mauricevanveen.com