search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.15k stars 3.81k forks source link

util/growstack: investigate Go 1.19 stack growth #88232

Open erikgrinaker opened 2 years ago

erikgrinaker commented 2 years ago

As seen in #88038, the upgrade to Go 1.19 caused a significant performance regression that was found to be due to the new stack growth behavior in Go 1.19. The release notes say:

The runtime will now allocate initial goroutine stacks based on the historic average stack usage of goroutines. This avoids some of the early stack growth and copying needed in the average case in exchange for at most 2x wasted space on below-average goroutines.

We already have pkg/util/growstack which will initialize stacks to at least 32 KB:

https://github.com/cockroachdb/cockroach/blob/c097a16427f65e9070991f062716d222ea5903fe/pkg/util/growstack/growstack.go#L13-L20

The fix for the performance regression was to double the initial stack size here, as seen in #88187.

We should investigate further why doubling the initial stack size helped, and whether we can avoid using the growstack package at all. One possibility is that the old behavior would double the stack size when growing it, while the new behavior possibly uses a smaller multiple, leading to more frequent growing.

We should also communicate this upstream to the Go team and see if the runtime needs improvements.

Jira issue: CRDB-19739

github-actions[bot] commented 8 months ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB!