sosy-lab / benchexec

BenchExec: A Framework for Reliable Benchmarking and Resource Measurement
Apache License 2.0
231 stars 199 forks source link

Question on usage of arguments #744

Closed timur-g closed 3 years ago

timur-g commented 3 years ago

Sorry for posting this, but I couldn't figure out how to use arguments.

Using backquotes around program and it's arguments (not see here) runexec ends with "usage" help and no measurements: runexec program --programoption /path/file.ext

This just starts, but stalls and doesn't run: runexec -- program --programoption /path/file.ext

PhilippWendler commented 3 years ago

If you want to pass arguments to the executed program, you must use -- to separate the arguments for runexec from the command line for the tool (that is a syntax also used by other tools).

If the started program stalls and seems to do nothing, this is a different problem and you would need to debug it. For example, maybe the program is actually started but does nothing? Try with something like -h to get output from program. Or maybe there is a bug in runexec. Then using runexec --debug might be useful. But this is not related to argument usage.

timur-g commented 3 years ago

Thanks and sorry for this lazy person approach. I tried this only today and didn't go into how it works. Here is debug output.

2021-08-26 13:07:30 - INFO - Writing output to output.log 2021-08-26 13:07:30 - DEBUG - Setting up cgroups for run. 2021-08-26 13:07:30 - DEBUG - Created cgroups {'/sys/fs/cgroup/memory/system.slice/benchexec-cgroup.service/benchmarkddr0fix', '/sys/fs/cgroup/cpu,cpuacct/system.slice/benchexec-cgroup.service/benchmark_2izua90w', '/sys/fs/cgroup/freezer/system.slice/benchexec-cgroup.service/benchmark_7pr4huq8', '/sys/fs/cgroup/blkio/system.slice/benchexec-cgroup.service/benchmark_jyriais1'}. 2021-08-26 13:07:30 - DEBUG - Using additional environment {}. 2021-08-26 13:07:30 - DEBUG - Starting process. 2021-08-26 13:07:30 - DEBUG - Parent: child process of RunExecutor with PID 5733 started. 2021-08-26 13:07:30 - DEBUG - Child: child process of RunExecutor with PID 5733 started 2021-08-26 13:07:30 - DEBUG - Failed to make b'/tmp/BenchExec_run_pl6m800o/mount/home/benchexec' a bind mount: [Errno 2] mount(b'/tmp/BenchExec_run_pl6m800o/mount/home/benchexec', b'/tmp/BenchExec_run_pl6m800o/mount/home/benchexec', None, 4096, None) failed: No such file or directory 2021-08-26 13:07:30 - DEBUG - Mounting '/' as overlay 2021-08-26 13:07:30 - DEBUG - [Errno 16] umount(b'/tmp/BenchExec_run_pl6m800o/mount/') failed: Device or resource busy 2021-08-26 13:07:30 - DEBUG - Creating overlay mount: target=b'/tmp/BenchExec_run_pl6m800o/mount/', lower=b'/', upper=b'/tmp/BenchExec_run_pl6m800o/temp/', work=b'/tmp/BenchExec_run_pl6m800o/overlayfs/' 2021-08-26 13:07:30 - DEBUG - Mounting '/dev' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/dev/pts' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/dev/shm' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/dev/mqueue' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/dev/hugepages' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/run' as hidden 2021-08-26 13:07:30 - DEBUG - [Errno 22] umount(b'/tmp/BenchExec_run_pl6m800o/mount/run') failed: Invalid argument 2021-08-26 13:07:30 - DEBUG - Mounting '/sys' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/kernel/security' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/unified' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/systemd' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/pids' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/devices' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/cpu,cpuacct' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/net_cls,net_prio' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/perf_event' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/rdma' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/blkio' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/hugetlb' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/freezer' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/memory' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/cpuset' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/pstore' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/kernel/debug' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/kernel/config' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/fuse/connections' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/proc' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/var/lib/lxcfs' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/path' as overlay 2021-08-26 13:07:30 - DEBUG - [Errno 22] umount(b'/tmp/BenchExec_run_pl6m800o/mount/path') failed: Invalid argument 2021-08-26 13:07:30 - DEBUG - Creating overlay mount: target=b'/tmp/BenchExec_run_pl6m800o/mount/path', lower=b'/path', upper=b'/tmp/BenchExec_run_pl6m800o/temp/path', work=b'/tmp/BenchExec_run_pl6m800o/overlayfs/path' 2021-08-26 13:07:30 - DEBUG - Mounting '/path' as overlay 2021-08-26 13:07:30 - DEBUG - [Errno 22] umount(b'/tmp/BenchExec_run_pl6m800o/path') failed: Invalid argument 2021-08-26 13:07:30 - DEBUG - Creating overlay mount: target=b'/tmp/BenchExec_run_pl6m800o/path', lower=b'/path', upper=b'/tmp/BenchExec_run_pl6m800o/temp/path', work=b'/tmp/BenchExec_run_pl6m800o/overlayfs/path' 2021-08-26 13:07:30 - DEBUG - Mounting '/' as overlay 2021-08-26 13:07:30 - DEBUG - [Errno 16] umount(b'/tmp/BenchExec_run_pl6m800o/mount/') failed: Device or resource busy 2021-08-26 13:07:30 - DEBUG - Creating overlay mount: target=b'/tmp/BenchExec_run_pl6m800o/mount/', lower=b'/', upper=b'/tmp/BenchExec_run_pl6m800o/temp/', work=b'/tmp/BenchExec_run_pl6m800o/overlayfs/' 2021-08-26 13:07:30 - DEBUG - Mounting '/dev' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/dev/pts' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/dev/shm' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/dev/mqueue' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/dev/hugepages' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/run' as hidden 2021-08-26 13:07:30 - DEBUG - [Errno 22] umount(b'/tmp/BenchExec_run_pl6m800o/mount/run') failed: Invalid argument 2021-08-26 13:07:30 - DEBUG - Mounting '/sys' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/kernel/security' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/unified' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/systemd' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/pids' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/devices' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/cpu,cpuacct' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/net_cls,net_prio' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/perf_event' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/rdma' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/blkio' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/hugetlb' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/freezer' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/memory' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/cgroup/cpuset' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/pstore' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/kernel/debug' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/kernel/config' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/sys/fs/fuse/connections' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/proc' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/var/lib/lxcfs' as read-only 2021-08-26 13:07:30 - DEBUG - Mounting '/media/sf_Downloads' as overlay 2021-08-26 13:07:30 - DEBUG - [Errno 22] umount(b'/tmp/BenchExec_run_pl6m800o/path') failed: Invalid argument 2021-08-26 13:07:30 - DEBUG - Creating overlay mount: target=b'/tmp/BenchExec_run_pl6m800o/mount/path', lower=b'/media/sf_Downloads', upper=b'/tmp/BenchExec_run_pl6m800o/temp/path', work=b'/tmp/BenchExec_run_pl6m800o/overlayfs/path' 2021-08-26 13:07:30 - DEBUG - Mounting '/path' as overlay 2021-08-26 13:07:30 - DEBUG - [Errno 22] umount(b'/tmp/BenchExec_run_pl6m800o/mount/path') failed: Invalid argument 2021-08-26 13:07:30 - DEBUG - Creating overlay mount: target=b'/tmp/BenchExec_run_pl6m800o/mount/path', lower=b'/path', upper=b'/tmp/BenchExec_run_pl6m800o/temp/path', work=b'/tmp/BenchExec_run_pl6m800o/overlayfs/path' 2021-08-26 13:07:30 - DEBUG - Mounting '/tmp' as hidden 2021-08-26 13:07:30 - DEBUG - [Errno 22] umount(b'/tmp/BenchExec_run_pl6m800o/mount/tmp') failed: Invalid argument 2021-08-26 13:07:30 - DEBUG - Mounting '/run' as hidden 2021-08-26 13:07:30 - DEBUG - Parent: executing instdir/program/soffice in grand child with PID 5734 via child with PID 5733. 2021-08-26 13:07:30 - DEBUG - Waiting for signals

timur-g commented 3 years ago

I tried with sudo, it doesn't hang but stops and says this:

Cannot start process: [Errno 13] Permission denied: 'program' Cannot execute 'instdir/program/soffice': execution in container failed, check log for details.

Not sure what permission it is.

PhilippWendler commented 3 years ago

Thanks and sorry for this lazy person approach. I tried this only today and didn't go into how it works. Here is debug output.

[...] 2021-08-26 13:07:30 - DEBUG - Parent: executing instdir/program/soffice in grand child with PID 5734 via child with PID 5733. 2021-08-26 13:07:30 - DEBUG - Waiting for signals

This looks just fine. I think that it is simply your program that is correctly executed but then does nothing (or appears do be doing nothing). And why this happens of course depends on your program.

Is soffice the office suite? Then I guess it will try to initialize a GUI and that would probably fail (although my soffice has different failure behavior in this situation). BenchExec is intended for non-interative and non-GUI programs, and creating a GUI from the container is unlikely to work (because we never tested it).

PhilippWendler commented 3 years ago

I tried with sudo, it doesn't hang but stops and says this:

Cannot start process: [Errno 13] Permission denied: 'program' Cannot execute 'instdir/program/soffice': execution in container failed, check log for details.

Not sure what permission it is.

Note that BenchExec is intended to be used as a regular user, and the techniques it uses will work properly only as a regular non-root user. Running it at root could mean that for example the containerization is not effective. So using sudo for BenchExec is strongly discouraged and never the correct solution for working around problems.

timur-g commented 3 years ago

Yes, this was about office GUI program so it obviously fails. I don't normally run sudo.