This PR refactors integration test startup and teardown so it supports clustered organizations in Buildkite, which is the majority.
Concretely, now each integration test will spin up an ephemeral agent queue + pipeline.
Combined with the improvement in development.md, hopefully this PR also unblocks members of public to execute our integration tests in their own k8s cluster.
Previously, nobody can run the integration unless they owns an unclustered Buildkite organization.
Beside the main theme, this PR also slightly increase the ergonomic of test setup by moving the cleanup invocation into the test helpers.
If this PR is merged, the next step is to migrate our CI org to clusterd org and potentially marking cluster-id as mandatory.
This PR refactors integration test startup and teardown so it supports clustered organizations in Buildkite, which is the majority.
Concretely, now each integration test will spin up an ephemeral agent queue + pipeline.
Combined with the improvement in
development.md, hopefully this PR also unblocks members of public to execute our integration tests in their own k8s cluster. Previously, nobody can run the integration unless they owns an unclustered Buildkite organization.Beside the main theme, this PR also slightly increase the ergonomic of test setup by moving the cleanup invocation into the test helpers.
If this PR is merged, the next step is to migrate our CI org to clusterd org and potentially marking cluster-id as mandatory.