How to specify the python interpreter path in benchexec?

[shengfeng]

When I use the python virtual environment such as

source ~/venv/activate
benchexec ...

Everything is ok. However, if I want to the python interpreter in /home/user/.local/bin/python and then we execute

benchexec ...

This Benchexec framework tells us that tool distribution was not found. We found the default python interpreter in /usr/local/python38/bin/python is called, and is not the python interpreter we want. The error message is as follows:

Traceback (most recent call last):
  File "/home/user/.local/bin/tool", line 4, in <module>
    __import__('pkg_resources').require('tool==0.0.0.dev0')
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    def _initialize_master_working_set():
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3234, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3263, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'tool==0.0.0.dev0' distribution was not found and is required by the application

The reason is that the wrong python interpreter(/usr/local/python38/bin/python) is called, and we want to use the correct python interpreter ( /home/user/.local/bin/python).

How can we solve the problem without using the python virtual environment?

[PhilippWendler]

The stacktrace and the error message do not seem to come from BenchExec, there is nothing in it that references BenchExec. Instead what seems to happen is that /home/user/.local/bin/tool is executed, but BenchExec would not typically do that (except if you tell it to benchmark this executable).

Can you please clarify what you are attempting? Are you trying to benchmark something called tool? If yes, it seems you need to fix its installation. BenchExec should not interfere in any way with what Python interpreter is used.

[shengfeng]

We develop a verification tool named veri-tool (a bad name) in Python, and the default tool path in /home/user/.local/bin/veri-tool. Then we integrate our verification tool in Benchexec guided by the document tool-integration. In fact, our integrated verification tool in Benchexec should be interpretered by the /home/user/.local/bin/python. However, when we execute benchexec veri-tool ..., we had a error message as follows:

Traceback (most recent call last):
  File "/home/user/.local/bin/veri-tool", line 4, in <module>
    __import__('pkg_resources').require('veri-tool==0.0.0.dev0')
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    def _initialize_master_working_set():
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3234, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3263, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/python38/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'veri-tool==0.0.0.dev0' distribution was not found and is required by the application

which means the wrong python interpreter(/usr/local/python38/bin/python) is called. But if we call our verification tool veri-tool directly using subprocess in python, it works well. Is it possible that the python interpreter under the HOME directory cannot be found after using cgroup？

[PhilippWendler]

So this stacktrace appears as part of the output of your tool? Not of BenchExec, correct?

cgroups have nothing to do with which files can be found, but containers do. Are you hiding any directories inside the container with --hidden-dir?

What is the content of /home/user/.local/bin/veri-tool?

What is your PATH environment variable?

[shengfeng]

Yes. This stacktrace is as part of the output of our tool, not of BenchExec. I add parameter --no-container when executing BenchExec.

The /home/user/.local/bin/veri-tool is the script of our tool, same as symbiotic and so on. My Path is as follows: /home/user/.opam/4.09.0/bin:/home/user/venv/bin:/home/user/.autojump/bin:/home/user/.cargo/bin:/home/user/.autojump/bin:/home/user/.nvm/versions/node/v16.16.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/user/.local/bin. It already contains /home/user/.local/bin, where the correct python interpreter is located.

When BenchExec calls our intergrated tool veri-tool, is effect the same as calling the script /home/user/.local/bin/veri-tool? When we invoke the script /home/user/.local/bin/veri-tool FILE directly in shell, it works well. However, when we call the veri-tool through BenchExec, it has an error message above, which means that the python interpreter cannot find our verification tool as anonther python interpreter is wrongly invoked.

[PhilippWendler]

Please provide the content of /home/user/.local/bin/veri-tool.

I add parameter --no-container when executing BenchExec.

This is crucial information in such cases, please always provide all parameters that you use when asking for help.

Please provide the output of type veri-tool and the function executable() of your tool-info module (the Python module you wrote for BenchExec).

[PhilippWendler]

Do you still have this problem?

[shengfeng]

Yes.

Do you still have this problem?

However, I cannot provide the source code of veri-tool for some reason. The function executable() is as follows:

def executable(self, tool_locator):
    return tool_locator.find_executable("veri-tool")

Besides, the container mode is also unavailable in my system, with the following error messages:

Error: Failed to configure container: [Errno 22] Creating overlay mount for '/' failed: Invalid argument. Please use other directory modes, for example '--read-only-dir /'.

The current solution for me is to run BenchExec in python virtual environment to avoid the above errors, since it works well in virtual environment with no conatiner.

I am not sure whether the --no-container affects the path of python interpreter, the all parameters is benchexec /path/NoOverflows-Other.xml --no-compress-result --no-container --timelimit 900s -o /path/output.

[PhilippWendler]

Note: I strongly recommend to not turn off container mode just because it does not work on your system with the default settings. You will loose a lot of features and reliability of your benchmarking.

In this case your kernel likely is affected by #776 and you simply need to use --read-only-dir / as recommended in the error message, plus maybe some option for using overlayfs where necessary.

Please provide at least the first few lines of /home/user/.local/bin/veri-tool. In all likelihood, this file is just a wrapper around your tool, and not the actual tool.

[PhilippWendler]

Do you still have this problem and would like to solve it? Then please provide the requested information.