search

sosy-lab / benchexec

BenchExec: A Framework for Reliable Benchmarking and Resource Measurement
Apache License 2.0
219 stars 191 forks source link

Command-line verbs for test_tool_info and check_cgroups #1053

Open PhilippWendler opened 2 weeks ago

PhilippWendler commented 2 weeks ago

Right now, to use our test_tool_info and check_cgroups functionality, users have to use commands like python3 -m benchexec.test_tool_info .... This is documented, but not easy to remember. We could make this easier by providing a direct entry point.

One way would be to drop more executables in the user's PATH, like test-benchexec-tool-info or so. But this is also hard to discover and clutters the global namespace.

A common way to have one executable with several distinct personalities / modes is to use tool verb args. We could use this and have something like benchexec test-tool-info .... It is somewhat uncommon to have a tool with some modes that use such verbs and one mode that does not use a verb (the standard benchmarking mode), but we can live with that.

For check_cgroups, both benchexec and runexec could gain that verb.

PhilippWendler commented 2 weeks ago

Doing this would also make test_tool_info work out-of-the-box again on Ubuntu 24.04 (cf. https://github.com/sosy-lab/benchexec/issues/1041#issuecomment-2165691605).