cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.18k stars 3.82k forks source link

CLI prints error and dumps on Ctrl-C #118143

Open rolfen opened 10 months ago

rolfen commented 10 months ago

Describe the problem

ran cockroach start-single-node and then hit Ctrl-C

Everytime it runs it leaves behind the following in the running directory:

And prints an error on stderr:

E240123 08:58:48.472333 2590 sql/sqlstats/persistedsqlstats/flush.go:84  [T1,n1,server drain process,call-graceful-drain] 4  encountered an error at flush, checking for statement statistics size limit: fetch-stmt-count: cannot acquire lease when draining
E240123 08:58:48.473924 3284 kv/kvserver/queue.go:1142  [T1,n1,split,s1,r64/1:/{Table/104-Max}] 5  unable to split [n1,s1,r64/1:/{Table/104-Max}] at key /Table/105: split at key /Table/105 failed: failed to send RPC: sending to all replicas failed; last error: node unavailable; try another peer
*
* INFO: interrupted
*
ERROR: interrupted
Failed running "start-single-node"

To Reproduce

cockroach start-single-node --insecure --store=type=mem,size=0.25 --advertise-addr=localhost --logtostderr=ERROR --background 

Expected behavior

Shutdown gracefully and keep the working dir clean.

Additional context

MacOs 12.7.2 cockroach version details: Build Tag: v23.1.13 Build Time: 2023/12/01 21:57:15 Distribution: CCL Platform: darwin amd64 (x86_64-apple-darwin19) Go Version: go1.19.13 C Compiler: Clang 10.0.0 Build Commit ID: 8d065df26e25a762a1abbbf2b1e475456b341a00 Build Type: release Enabled Assertions: false

Jira issue: CRDB-35568

blathers-crl[bot] commented 10 months ago

Hello, I am Blathers. I am here to help you get the issue triaged.

Hoot - a bug! Though bugs are the bane of my existence, rest assured the wretched thing will get the best of care here.

I was unable to automatically find someone to ping.

If we have not gotten back to your issue within a few business days, you can try the following:

:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

stevendanna commented 10 months ago

Thanks for filing the report.

It appears that the output directory for the various runtime dumps end up in the current working directory if there is no store or log directory configured:

https://github.com/cockroachdb/cockroach/blob/ba52733b8f92c17e8902de81591debf04d0fb076/pkg/cli/log_flags.go#L211-L227

As for the most sensible default behaviour when the user has taken such clear attempts to have an in-memory database, I think possible options include disabling these dumps completely, using a temporary directory as the fallback default, or providing some user-control over the creation and placement of these files. I'm not sure we want to clean these up on exit since the whole point of these is often to do post-mortem analysis of a problem that might have resulted in a server exit.