search

DOMjudge / domjudge-packaging

DOMjudge packaging for (Linux) distributions and live image
31 stars 37 forks source link

Docker Swarm incompatiblity #48

Closed rmartinsanta closed 4 years ago

rmartinsanta commented 4 years ago

Using a privileged container (the judgehost) makes the docker-compose.yml incompatible with Docker Swarm, why does it exactly need to be privileged? Are there any alternatives to make it work without using privileged: true?

Thanks in advance

meisterT commented 4 years ago

We're using cgroups for sandboxing/security in the judgehosts. As far as I know, we have to use privileged in order to access cgroups from within docker. Let us know if there's any better solution.

rmartinsanta commented 4 years ago

I tried with capabilities but they they arent supported yet by Docker Swarm, this PR looks promising, I will test it and report back https://github.com/docker/cli/pull/1940.

Thanks for your time.

rmartinsanta commented 4 years ago

In the latest build privileged true is supposed to be supported, however when sending a submission I get the following error in the judgehost:

[Dec 11 16:53:52.203] judgedaemon[57]: Fetching new executable 'cpp' [Dec 11 16:53:53.917] judgedaemon[57]: executing chroot script: 'chroot-startstop.sh start' mount: /opt/domjudge/judgehost/judgings/51ffe0c344ce-0/endpoint-default/2/1/2/proc: bind /proc failed. [Dec 11 16:53:53.923] judgedaemon[57]: error: chroot script exited with exitcode 32

Any idea what is missing from the judgehost?

nickygerritsen commented 4 years ago

Hmmm no, I have not seen this before

eldering commented 4 years ago

I don't know, but it seems the bind mount of /proc fails. You can try to debug it by running it manually in your docker instance and see if you get any clearer error message. The script that calls it https://github.com/DOMjudge/domjudge/blob/master/judge/chroot-startstop.sh.in is quite self-explanatory.

rmartinsanta commented 4 years ago

The error can be reproduced in the nightly Docker build in Swarm mode by executing the following command inside the judgehost (or equivalent, as chroot-startstop.sh does): mount -t proc /proc proc/ which gives the error image This is probably a bug in the nightly docker engine or in my docker swarm set up, so do not invest time looking into it.

I managed to get docker-compose fully working though, I will make a pull request after I finish polishing it a bit, I think it can simplify deployment and operations.