search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.85k stars 3.77k forks source link

Periodically dump goroutines along with memory profiles #52814

Open tim-o opened 4 years ago

tim-o commented 4 years ago

Is your feature request related to a problem? Please describe. We frequently run into cases where we can see a problem occurred in the database in our metrics, but are unable to identify a culprit in the logs. In these cases, the guidance is often to capture a CPU profile, memory profile, and goroutine dump while the issue is happening.

Describe the solution you'd like Thanks to @knz's recent work, we'll now retain heap profiles that are generated periodically. It would be helpful if we could collect periodic goroutine dumps as well.

Describe alternatives you've considered We will collect dumps if the count of goroutines increases significantly. However, we also see cases where goroutines get "stuck", while the count remains more or less the same.

Jira issue: CRDB-3912

blathers-crl[bot] commented 4 years ago

Hi @tim-o, I've guessed the C-ategory of your issue and suitably labeled it. Please re-label if inaccurate.

While you're here, please consider adding an A- label to help keep our repository tidy.

:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

github-actions[bot] commented 1 year ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB!