search

DOMjudge / domjudge-packaging

DOMjudge packaging for (Linux) distributions and live image
31 stars 37 forks source link

Dockerized judgehost failed on program run #33

Closed vmmelnikov closed 5 years ago

vmmelnikov commented 5 years ago

Hi! I see this message on judging every program:

/opt/domjudge/judgehost/bin/runguard: cannot start /testcase001/execdir/program': Resource temporarily unavailable Try/opt/domjudge/judgehost/bin/runguard --help' for more information.

Compilation is OK Version is 6.0.3

nickygerritsen commented 5 years ago

Interesting. Do you have any special setup for Docker? I have not run into this yet

vmmelnikov commented 5 years ago

Nothing special. Ubuntu 18.04_2 LTS (as host OS) + docker For the some Ubuntu in VirtualBox works fine

nickygerritsen commented 5 years ago

Could you list the steps you did, so I can reproduce this?

vmmelnikov commented 5 years ago

  1. Environment Host OS: 18.04.2 LTS (Bionic Beaver) Docker version 18.09.2, build 6247962 docker-compose version 1.24.0, build 0aa59064

    cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-4.15.0-47-generic root=UUID=b948634b-8e4c-4805-9327-53aad87782d8 ro alx.enable_wol=1 mem_sleep_default=deep quiet cgroup_enable=memory swapaccount=1 splash vt.handoff=1

  2. Start

    docker-compose -f domjudge.yaml up -d (several times until judgehost is up) cat domjudge.yaml version: "3.2" services: dj-mariadb: image: mariadb:10.2 ports:

    • "13306:3306" environment: MYSQL_ROOT_PASSWORD: r54d9Kl2 MYSQL_USER: domjudge MYSQL_PASSWORD: djpwH6f34e MYSQL_DATABASE: domjudge networks:
    • default volumes:
    • "/var/lib/domjudge/db:/var/lib/mysql" domserver: image: domjudge/domserver:6.0.3 ports:
    • "80:80" environment: CONTAINER_TIMEZONE: Europe/Moscow MYSQL_HOST: mariadb MYSQL_USER: domjudge MYSQL_DATABASE: domjudge MYSQL_PASSWORD: djpwH6f34e MYSQL_ROOT_PASSWORD: r54d9Kl2 networks:
    • default volumes:
    • "/sys/fs/cgroup:/sys/fs/cgroup:ro" links:
    • dj-mariadb:mariadb judgehost-0: image: domjudge/judgehost:6.0.3 hostname: mainjudge privileged: true environment: CONTAINER_TIMEZONE: Europe/Moscow DAEMON_ID: 0 JUDGEDAEMON_USERNAME: judgehost JUDGEDAEMON_PASSWORD: fGratd56rQk98Ret2m0PG6 networks:
    • default volumes:
    • "/sys/fs/cgroup:/sys/fs/cgroup:ro" links:
    • domserver:domserver

  3. Create user that have roles admin, jury, team

  4. Submit 'Hello world' solution from team screen

  5. Return to jury screen/submissions Program error output /opt/domjudge/judgehost/bin/runguard: cannot start /testcase001/execdir/program': Resource temporarily unavailable Try/opt/domjudge/judgehost/bin/runguard --help' for more information.

nickygerritsen commented 5 years ago

Thanks! We can indeed reproduce this and are investigating the issue and a possible solution.

nickygerritsen commented 5 years ago

The problem lies in the fact that we use UID/GID 1001 in the judgehost container and limit the number of processes for that user. However, the chance that that user exists on your Ubuntu host OS is very high and in that case the processes from that user will also be counted towards the limit of (by default) 64 processes. I have just pushed new versions of the 6.0.3 and 5.3.3 judgehost images to Docker Hub, where we will use UID/GID 62860 by default, which can be overriden with the RUN_USER_UID_GID environment variable. I have also updated the readme on Docker Hub. This will fix the issue you are having.

In the future we might want to use the cgroup pid namespace controller, which should prevent this issue. But for now the above method is a decent enough fix.