Very difficult usage of the handler in Virgo

[RenHirayama]

I'm trying to use it in the cluster with @EggerJoscha but currently it's very hard.

The problems we found:

• we must use a smash container to open a virtual environment to install pip to download requirements (some that are not in python/requirements.txt), instead of just having a container with the necessary requirements
• numpy 2.1.1 is not accepted even though it requires 1.11
• several unit tests fail, see image below (functional tests work)
• the error message is not helpful (e.g. "Fatal: IC run failed"). We found the logs after solving the issue, but it'd be helpful if the message warned us that there was a log
• "column" is not available in the container but this didn't interfere in anything so far

tests_runner.log

It seems that the original problem is a Pythia installation, but we took a longer time to realize this and I think the manager could've helped us. We got it working with a workaround from @RobinSattler, but at least we should have info on how to do it in our internal GitLab wiki.

I think we need to improve this side though, as the hybrid is mostly done in HPC

[AxelKrypton]

Before having a closer look, which I will do at some point, let me stress a couple of aspects (maybe we should check if they are stressed enough in the user guide, too).

• The hybrid handler is nothing more than a big script that runs in user's terminal, which implies that the user's environment has to be properly set up to allow the handler to work as intended. We cannot really get around this aspect, but this is also why we invested so much time about offering a handy interface and documentation about system requirements. The user should first run the handler to get the requirements overview and then adjust the environment, if needed. Of course, we can ship a container to offer a ready environment, but we decided to not do this, yet. After all GNU commands and python requirements should be simple enough to be handled by the average user. Furthermore, there is the complication that user's executables might have to be run in a container (this is the case for SMASH) and then one might be confused about what to do having different containers.
• All tests should pass before starting using the container. Having a look to the log, it seems that most (if not all) failing ones are failing because of the environment. However, the statement about Python is surprising. Which requirements were needed, which we did not include as such? This might need some improvement, indeed.
• We require numpy~=1.11 that means any version compatible with version 1.11 and version 2.x are not as version 2 is not backward compatible with version 1. We need to further discuss this and take a decision.
• I am not surprised that a missing column command is not critical. This will mainly affect how output is formatted, probably one can survive with this, but it is probably nicer to leave it in as requirement.
• We can change "Fatal: IC run failed" to "Fatal: IC run failed (look at its output files)" to be more explicit. 👍

[AxelKrypton]

We decided to provide the user with an Ubuntu-based container in which it should be straightforward to install all software needed by the Handler (Pythia, SMASH, vHLLE). I will take care of preparing it within this issue addressing then the points above and reporting once done.