search

google-deepmind / code_contests

Apache License 2.0
2.1k stars 208 forks source link

Executing and evaluating solutions 3x Compilation failure with many python versions #31

Open TheExGenesis opened 1 year ago

TheExGenesis commented 1 year ago

Default python:

(base) root@os-nvme-o49i23bw-cpu-4v-fin1:~/workspace/code_contests# which python
/root/anaconda3/bin/python

(base) root@os-nvme-o49i23bw-cpu-4v-fin1:~/workspace/code_contests# bazel run -c opt execution:solve_example --   --valid_path=/tmp/dm-code_contests/code_contests_valid.riegeli
INFO: Analyzed target //execution:solve_example (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //execution:solve_example up-to-date:
  bazel-bin/execution/solve_example
INFO: Elapsed time: 0.106s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/execution/solve_example '--valid_path=/tmp/dm-code_contests/code_contests_valid.riegeli'
We will try to solve "Gregor and Cryptography":
https://codeforces.com/problemset/problem/1549/A

We will run:
  1. A program that does not compile.
  2. A program that runs successfully, but gives the wrong answer sometimes.
  3. A correct solution.

--------------------------------------------------------------------------------
An invalid program is reported as not compiling:

[global_forkclient.cc : 122] RAW: Starting global forkserver
WARNING: Logging before InitGoogleLogging() is written to STDERR
E20230316 15:58:35.383378 1229811 monitor.cc:842] SANDBOX VIOLATION : PID: 1229816, PROG: 'python3.9' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7ffd16369bd0) IP: 0x7efc528b2bf5, STACK: 0x7ffd16369b80
I20230316 15:58:35.383520 1229811 monitor.cc:386] Stack traces have been disabled
Compilation failed

--------------------------------------------------------------------------------
The bad solution passes a few tests but then fails.
Because we set stop_on_first_failure to True, we stop once we see a failure.
We are running on 4 threads, so it's possible that more than one failure occurs
before all threads stop.

E20230316 15:58:35.538933 1229818 monitor.cc:842] SANDBOX VIOLATION : PID: 1229821, PROG: 'python3.9' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7ffc6307a710) IP: 0x7fc096ca8bf5, STACK: 0x7ffc6307a6c0
I20230316 15:58:35.539032 1229818 monitor.cc:386] Stack traces have been disabled
Compilation failed

--------------------------------------------------------------------------------
The good solution passes all tests.

E20230316 15:58:35.697964 1229824 monitor.cc:842] SANDBOX VIOLATION : PID: 1229827, PROG: 'python3.9' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7fff542e0e00) IP: 0x7f0b74a0fbf5, STACK: 0x7fff542e0db0
I20230316 15:58:35.698267 1229824 monitor.cc:386] Stack traces have been disabled
Compilation failed

anaconda python 3.9

(base) root@os-nvme-o49i23bw-cpu-4v-fin1:~/workspace/code_contests# bazel run -c opt execution:solve_example --   --valid_path=/tmp/dm-code_contests/code_contests_valid.riegeli   --python3_path=/root/anaconda3/bin/python3.9 --python3_library_paths=/root/anaconda3/lib/python3.9
INFO: Analyzed target //execution:solve_example (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //execution:solve_example up-to-date:
  bazel-bin/execution/solve_example
INFO: Elapsed time: 0.083s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/execution/solve_example '--valid_path=/tmp/dm-code_contests/code_contests_valid.riegeli' '--python3_path=/root/anaconda3/bin/python3.9' '--python3_library_paths=/root/anaconda3/lib/python3.9'
We will try to solve "Gregor and Cryptography":
https://codeforces.com/problemset/problem/1549/A

We will run:
  1. A program that does not compile.
  2. A program that runs successfully, but gives the wrong answer sometimes.
  3. A correct solution.

--------------------------------------------------------------------------------
An invalid program is reported as not compiling:

[global_forkclient.cc : 122] RAW: Starting global forkserver
WARNING: Logging before InitGoogleLogging() is written to STDERR
E20230316 16:00:09.580668 1230504 monitor.cc:842] SANDBOX VIOLATION : PID: 1230509, PROG: 'python3.9' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7ffe0238a890) IP: 0x7f95de0dbbf5, STACK: 0x7ffe0238a840
I20230316 16:00:09.580809 1230504 monitor.cc:386] Stack traces have been disabled
Compilation failed

--------------------------------------------------------------------------------
The bad solution passes a few tests but then fails.
Because we set stop_on_first_failure to True, we stop once we see a failure.
We are running on 4 threads, so it's possible that more than one failure occurs
before all threads stop.

E20230316 16:00:09.735759 1230511 monitor.cc:842] SANDBOX VIOLATION : PID: 1230514, PROG: 'python3.9' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7ffe6356e7b0) IP: 0x7f551c435bf5, STACK: 0x7ffe6356e760
I20230316 16:00:09.735826 1230511 monitor.cc:386] Stack traces have been disabled
Compilation failed

--------------------------------------------------------------------------------
The good solution passes all tests.

E20230316 16:00:09.890873 1230516 monitor.cc:842] SANDBOX VIOLATION : PID: 1230519, PROG: 'python3.9' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7fff51e6c1e0) IP: 0x7fc6584dcbf5, STACK: 0x7fff51e6c190
I20230316 16:00:09.890983 1230516 monitor.cc:386] Stack traces have been disabled
Compilation failed

python 3.8

(base) root@os-nvme-o49i23bw-cpu-4v-fin1:~/workspace/code_contests# bazel run -c opt execution:solve_example --   --valid_path=/tmp/dm-code_contests/code_contests_valid.riegeli   --python3_path=/usr/bin/python3.8 --python3_library_paths=/usr/lib/python3.8
INFO: Analyzed target //execution:solve_example (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //execution:solve_example up-to-date:
  bazel-bin/execution/solve_example
INFO: Elapsed time: 0.086s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/execution/solve_example '--valid_path=/tmp/dm-code_contests/code_contests_valid.riegeli' '--python3_path=/usr/bin/python3.8' '--python3_library_paths=/usr/lib/python3.8'
We will try to solve "Gregor and Cryptography":
https://codeforces.com/problemset/problem/1549/A

We will run:
  1. A program that does not compile.
  2. A program that runs successfully, but gives the wrong answer sometimes.
  3. A correct solution.

--------------------------------------------------------------------------------
An invalid program is reported as not compiling:

[global_forkclient.cc : 122] RAW: Starting global forkserver
WARNING: Logging before InitGoogleLogging() is written to STDERR
E20230316 16:01:16.592478 1231347 monitor.cc:842] SANDBOX VIOLATION : PID: 1231352, PROG: 'python3.8' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7ffc04acc560) IP: 0x7f66f5082bf5, STACK: 0x7ffc04acc510
I20230316 16:01:16.592905 1231347 monitor.cc:386] Stack traces have been disabled
Compilation failed

--------------------------------------------------------------------------------
The bad solution passes a few tests but then fails.
Because we set stop_on_first_failure to True, we stop once we see a failure.
We are running on 4 threads, so it's possible that more than one failure occurs
before all threads stop.

E20230316 16:01:16.744000 1231354 monitor.cc:842] SANDBOX VIOLATION : PID: 1231357, PROG: 'python3.8' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7ffe9b9ba820) IP: 0x7f49bca28bf5, STACK: 0x7ffe9b9ba7d0
I20230316 16:01:16.744094 1231354 monitor.cc:386] Stack traces have been disabled
Compilation failed

--------------------------------------------------------------------------------
The good solution passes all tests.

E20230316 16:01:16.897409 1231359 monitor.cc:842] SANDBOX VIOLATION : PID: 1231362, PROG: 'python3.8' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7ffdf57544b0) IP: 0x7f4901a41bf5, STACK: 0x7ffdf5754460
I20230316 16:01:16.897512 1231359 monitor.cc:386] Stack traces have been disabled
Compilation failed

python 2.7

root@os-nvme-o49i23bw-cpu-4v-fin1:~/workspace/code_contests# bazel run -c opt execution:solve_example --   --valid_path=/tmp/dm-code_contests/code_contests_valid.riegeli   --python2_path=/usr/bin/python2.7 --python2_library_paths=/usr/lib/python2.7
INFO: Analyzed target //execution:solve_example (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //execution:solve_example up-to-date:
  bazel-bin/execution/solve_example
INFO: Elapsed time: 0.080s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/execution/solve_example '--valid_path=/tmp/dm-code_contests/code_contests_valid.riegeli' '--python2_path=/usr/bin/python2.7' '--python2_library_paths=/usr/lib/python2.7'
We will try to solve "Gregor and Cryptography":
https://codeforces.com/problemset/problem/1549/A

We will run:
  1. A program that does not compile.
  2. A program that runs successfully, but gives the wrong answer sometimes.
  3. A correct solution.

--------------------------------------------------------------------------------
An invalid program is reported as not compiling:

[global_forkclient.cc : 122] RAW: Starting global forkserver
WARNING: Logging before InitGoogleLogging() is written to STDERR
E20230316 16:01:52.078231 1231946 monitor.cc:842] SANDBOX VIOLATION : PID: 1231954, PROG: 'python3.9' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7ffd73044620) IP: 0x7fd3674c1bf5, STACK: 0x7ffd730445d0
I20230316 16:01:52.078379 1231946 monitor.cc:386] Stack traces have been disabled
Compilation failed

--------------------------------------------------------------------------------
The bad solution passes a few tests but then fails.
Because we set stop_on_first_failure to True, we stop once we see a failure.
We are running on 4 threads, so it's possible that more than one failure occurs
before all threads stop.

E20230316 16:01:52.236987 1231961 monitor.cc:842] SANDBOX VIOLATION : PID: 1231964, PROG: 'python3.9' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7ffee2bab2c0) IP: 0x7f3327f7fbf5, STACK: 0x7ffee2bab270
I20230316 16:01:52.237118 1231961 monitor.cc:386] Stack traces have been disabled
Compilation failed

--------------------------------------------------------------------------------
The good solution passes all tests.

E20230316 16:01:52.387142 1231966 monitor.cc:842] SANDBOX VIOLATION : PID: 1231969, PROG: 'python3.9' : [X86-64] arch_prctl [158](0x3001 [12289], 0x7fffe265f0f0) IP: 0x7f731a7fcbf5, STACK: 0x7fffe265f0a0
I20230316 16:01:52.387249 1231966 monitor.cc:386] Stack traces have been disabled
Compilation failed
felixgimeno commented 1 year ago

It looks like for some reason the syscall arch_prctl is needed can you try adding said syscall to this line and re-executing these? It should solve this error

builder.AllowSyscall(__NR_arch_prctl);

https://github.com/deepmind/code_contests/blob/fa7a4f8139aab08362503f3344778eb86901709a/execution/py_tester_sandboxer.cc#L181

assapin commented 1 year ago

I also encountered this compilation failure with Python3.9. Adding the builder.AllowSyscall(__NR_arch_prctl); didn't work - I still got compilation issues in the example.

The only thing that solved it was to use Python 3.10 (in my case 3.10.13)