search

EmbarkStudios / k8s-buildkite-plugin

Run any buildkite build step as a Kubernetes Job
https://embark.dev
Apache License 2.0
49 stars 19 forks source link

Repo hooks are executed outside k8s job #31

Open keith opened 3 years ago

keith commented 3 years ago

When using buildkite's repo hooks as described here https://buildkite.com/docs/agent/v3/hooks#available-hooks if you have a post-checkout or pre-command hook, they are run outside of the k8s job that actually ends up running your code. This fails in a few ways for us since we use those to do some setup that depends on tools that are in our custom docker image, and to export environment variables we expect to exist later.

I believe this is because the core command is not run by buildkite-agent but instead is run directly https://github.com/EmbarkStudios/k8s-buildkite-plugin/blob/5826cbd20a4bd91e602c7a9bba58fd421e45d6c5/lib/job.jsonnet#L285-L286

I'm sure there was a good reason but I wonder if we could somehow run this with the buildkite-agent bootstrap command, which I assume would fix the issue? https://github.com/EmbarkStudios/k8s-buildkite-plugin/blob/5826cbd20a4bd91e602c7a9bba58fd421e45d6c5/lib/job.jsonnet#L321

keith commented 3 years ago

Note that buildkite-agent bootstrap supports --phases to do potentially less work. This could mean, depending on the background of that container being separate, we could change that command to run less and also run the second command through buildkite-agent

iffyio commented 3 years ago

I'm wondering if this is already possible, by default the buildkite agent binary is available to the job, thinking it should be possible to run bootstrap as the job's command instead - would those work in this case?

keith commented 3 years ago

Interesting! So I guess we could override that nice, I'm not sure if that's still how it should work though?

iffyio commented 3 years ago

Yeah it might not be, I'd say we give it a try first and if it'll be significantly simpler to have the plugin run via buidlkite-agent we can look into that - I can't really think of any disadvantages atm with running via the agent beyond the command's logs including buildkite output which should be fine (especially if we make it an opt-in feature)