search

bugcrowd / test-summary-buildkite-plugin

Buildkite plugin to summarise all test failures as an annotation
MIT License
68 stars 39 forks source link

Use with dockerized buildkite-agent #30

Open paulfri opened 6 years ago

paulfri commented 6 years ago

The latest buildkite-agent docker container uses docker 17.12 (the one on the ubuntu 18.04 repos) which does not have the --mount flag. NG, because this plugin uses the --mount flag. I guess I could file this issue with buildkite but I know you better.

https://github.com/bugcrowd/test-summary-buildkite-plugin/blob/74831d6a110d74af25673356c3eb82389e125e5f/hooks/command#L32

n.b. I have no idea if I can get this to work inside the container anyway, but this is a first step.

tessereth commented 6 years ago

Hey, nice to hear from you again!

I'll check it out. Probably need to go read some docker changelogs or something...

tessereth commented 6 years ago

Looks like --mount was added in 17.05 (https://docs.docker.com/release-notes/docker-ce/#17050-ce-2017-05-04). Are you sure that's actually the issue? Browsing some of the forks of this plugin and possibly related to #23, I think which buildkite-agent may not report correctly if you use buildkite-agent in a container. We don't do that so I have no means of testing it and I'm not entirely sure how that configuration functions. How does the buildkite-agent docker container even create other docker containers anyway? And how would one container have access to binaries in another one?

If you can find a way to make the buildkite-agent executable accessible inside the the plugin docker container then sweet, pls open PR. Otherwise I'm not sure how much I can help?

paulfri commented 6 years ago

Looks like --mount was added in 17.05 (https://docs.docker.com/release-notes/docker-ce/#17050-ce-2017-05-04).

Sorry, I got my numbers mixed up. From a shell in a running buildkite-agent container in our kube cluster:

root@buildkite-agent-75984756bf-88fxf:/# docker -v
Docker version 17.03.2-ce, build f5ec1e2
root@buildkite-agent-75984756bf-88fxf:/# docker run --mount
unknown flag: --mount
See 'docker run --help'

How does the buildkite-agent docker container even create other docker containers anyway? And how would one container have access to binaries in another one?

This is a thing that works™, it's probably not great but I intended to get further along before applying a ton of thought to it. The answer is the same as what you're doing here, right? Your Dockerfile doesn't have the agent in it, it's running on the host machine.

I dug around the @buildkite-plugins organization trying to find examples of other plugins that ran as docker containers but didn't see any. The buildkite-agent container doesn't have ruby on it, so updating the command hook to just (optionally?) run the plugin natively wouldn't work either, although maybe that's a better path to go down than nested container madness?

tessereth commented 6 years ago

So I checked and when you run the agent inside a container, you're supposed to mount the docker socket into it, thus allowing the agent container to create more containers (https://buildkite.com/docs/agent/v3/docker#invoking-docker-from-within-a-build). That all works fine assuming you have no dependencies but I can't think of any sensible way to then make the buildkite-agent binary in container A accessible in container B. You can mount files/directories from the host but that doesn't really help in this instance. I'll continue mulling about it in the background but ideas welcome.

As for supporting old docker versions... I mean it's easy enough to change --mount to -v. And if we manage to solve the other problems and that would unblock using this plugin with dockerized buildkite, then I'm happy to do it. But really buildkite should update their image. That version is old enough that it's got to be vulnerable to something.

tessereth commented 6 years ago

Maybe I'll experiment with adding the buildkite-agent binary directly into this container rather than mounting it. Not sure if having different agent versions managing the build and downloading artifacts would cause something to die horribly or not but could be worth trying. Means I'd have to update this container any time buildkite releases a version though, which sucks.