buildkite / agent-stack-k8s

Spin up an autoscaling stack of Buildkite Agents on Kubernetes
MIT License
77 stars 30 forks source link

Fix flaky panic in tests that log after test completion #289

Closed DrJosh9000 closed 5 months ago

DrJosh9000 commented 5 months ago

monitor.go:128 was logging after some tests had returned, which Go catches and helpfully panics for us. The logged error indicates context cancellation, which we can check for, and if that's the case just return.

If getScheduledCommandJobs returns errors repeatedly, then the continue makes the loop busy because getScheduledCommandJobs is called at the top. So I relocated the select to the top. To preserve getScheduledCommandJobs being called right away in the first iteration, there's a channel that can be read immediately on the first iteration.

Because there is now more than one return in the goroutine that is not also below a send on errs, I've also made close(errs) deferred in the goroutine.