consistent job crash

[qn06142]

Version: Latest version from docker Version string grabbed from page: 0.7.4 "Maaya Awatsuki" Bug: On submission, jobs fails with "exit code 2" some logs:

2024/09/13 14:52:41 [WORKER] Handling job failed: exit status 2
github.com/natsukagami/kjudge/worker/isolate.(*Sandbox).Run
        /kjudge/worker/isolate/sandbox.go:63
github.com/natsukagami/kjudge/worker.RunSingleCommand
        /kjudge/worker/run.go:122
github.com/natsukagami/kjudge/worker.Run
        /kjudge/worker/run.go:187
github.com/natsukagami/kjudge/worker.(*Queue).HandleJob
        /kjudge/worker/queue.go:75
github.com/natsukagami/kjudge/worker.(*Queue).Start
        /kjudge/worker/queue.go:37
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1571
github.com/natsukagami/kjudge/worker.RunSingleCommand
        /kjudge/worker/run.go:124
github.com/natsukagami/kjudge/worker.Run
        /kjudge/worker/run.go:187
github.com/natsukagami/kjudge/worker.(*Queue).HandleJob
        /kjudge/worker/queue.go:75
github.com/natsukagami/kjudge/worker.(*Queue).Start
        /kjudge/worker/queue.go:37
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1571
2024/09/13 14:52:41 [WORKER] Running submission 5 on [test `02`, group `main`]
2024/09/13 14:52:41 [WORKER] Handling job failed: exit status 2
github.com/natsukagami/kjudge/worker/isolate.(*Sandbox).Run
        /kjudge/worker/isolate/sandbox.go:63
github.com/natsukagami/kjudge/worker.RunSingleCommand
        /kjudge/worker/run.go:122
github.com/natsukagami/kjudge/worker.Run
        /kjudge/worker/run.go:187
github.com/natsukagami/kjudge/worker.(*Queue).HandleJob
        /kjudge/worker/queue.go:75
github.com/natsukagami/kjudge/worker.(*Queue).Start
        /kjudge/worker/queue.go:37
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1571
github.com/natsukagami/kjudge/worker.RunSingleCommand
        /kjudge/worker/run.go:124
github.com/natsukagami/kjudge/worker.Run
        /kjudge/worker/run.go:187
github.com/natsukagami/kjudge/worker.(*Queue).HandleJob
        /kjudge/worker/queue.go:75
github.com/natsukagami/kjudge/worker.(*Queue).Start
        /kjudge/worker/queue.go:37
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1571
2024/09/13 14:52:41 [WORKER] Submission 5 needs to run 2 tests before being scored.
2024/09/13 14:52:41 [WORKER] Running submission 5 on [test `01`, group `main`]

docker command: sudo docker run --privileged -d --rm --name kjudge --volume=kjudge:/data -p 8080:80 -e ADMIN_KEY=supersecret natsukagami/kjudge:latest

behavior: Make a contest. Add some problems Submit code for any problem Wait forever for submission to be judged

[natsukagami]

Can you provide us with a minimal set of test and solution code for your problem so we can reproduce it?

--

@minhnhatnoe can you take a look at this? exit code 2 sounds like some missing command (isolate?)

[qn06142]

Can you provide us with a minimal set of test and solution code for your problem so we can reproduce it?

--

@minhnhatnoe can you take a look at this? exit code 2 sounds like some missing command (isolate?)

I used the tests from this database: https://github.com/natsukagami/kjudge/files/4826816/kjudge.db.zip Also, I tried this set of tests, and solution, before trying the above db: https://drive.google.com/file/d/1xaVpZtQk6cZk3QJwjdRKGuy-zHObVYB5/view?usp=sharing https://drive.google.com/file/d/1dv7arAnMO0iOIV1VwArtDuaGQ7r-JH7b/view?usp=sharing (note: the two files were taken from free contest)

[natsukagami]

Thanks!

[qn06142]

A little update: I forgot to shutdown the server, the jobs are still coming in...

[natsukagami]

That's fine, jobs re-queue themselves on failure :)

[minhnhatnoe]

I'll take a look later today

[minhnhatnoe]

@qn06142 could you please send me the output of the command mount -v | grep cgroup on your host system (the system running the Docker daemon)? This might be due to your system running Control Groups v2, which support is currently being added in https://github.com/natsukagami/kjudge/pull/112

[qn06142]

mount -v | grep cgroup

It is indeed cgroups2, the command output is:

wheatley@wheatley-OptiPlex-7010:~$ mount -v | grep cgroup
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)

[qn06142]

side note: are there any quick workarounds? I am only using this for myself or people I trust, so sandbox is not needed.

[natsukagami]

@qn06142 with #112 merged, perhaps running the unstable branch would work, please try it out; we probably still have to iron it out before putting out a tagged released though.

[qn06142]

@qn06142 with #112 merged, perhaps running the unstable branch would work, please try it out; we probably still have to iron it out before putting out a tagged released though.

alright, trying that out right now, thanks :3

[qn06142]

yup, using this: sudo docker run --privileged -d --rm --name kjudge --volume=kjudge:/data -p 8080:80 -e ADMIN_KEY=secret natsukagami/kjudge:unstable jobs now finish normally, for some reason the version is still stuck on 0.7.4, though

[natsukagami]

That's fine ^^

[qn06142]

Should I close the issue or keep it open until 0.8.0 comes around?

[natsukagami]

Let's keep it open, we'll close it once we release 0.8. Thanks!

[qn06142]

Thanks for helping too ^_^