search

easybuilders / easybuild

EasyBuild - building software with ease
http://easybuild.io
GNU General Public License v2.0
464 stars 143 forks source link

One or more OS dependencies were not found #758

Closed kelbstf closed 2 years ago

kelbstf commented 2 years ago

Hello,

i try to install a toolchain, which (independently from the version) always fails:

eb --debug foss-2020a.eb --optarch=x86-rome --robot
== Temporary log file in case of crash /tmp/eb-zgiz6k7w/easybuild-ac71m4_n.log
== found valid index for /opt/ohpc/admin/easybuild/easybuild/easyconfigs, so using it...
== found valid index for /opt/ohpc/admin/easybuild/easybuild/easyconfigs, so using it...
ERROR: Failed to process easyconfig /opt/ohpc/admin/easybuild/easybuild/easyconfigs/f/foss/foss-2020a.eb: Failed to process easyconfig /opt/ohpc/admin/easybuild/easybuild/easyconfigs/o/OpenMPI/\
OpenMPI-4.0.3-GCC-9.3.0.eb: \
Failed to process easyconfig \
/opt/ohpc/admin/easybuild/easybuild/easyconfigs/u/UCX/UCX-1.8.0-GCCcore-9.3.0.eb:\
One or more OS dependencies were not found: \
[('libibverbs-dev', 'libibverbs-devel', 'rdma-core-devel')]

It's true: "libibverbs-dev', 'libibverbs-devel', 'rdma-core-devel" are not installed on OS level. And i would like to keep it like that, because i want to install the entire softwarestack using EasyBuild. Is it possible, and then how, to advise EasyBuild to simply install these dependencies on it's own, in order to make this software stack completely independent from the OS system libraries? That seems to be the best solution from my perspective.

Best regards

kelbstf commented 2 years ago

I did not mention why i wanted to avoid to install those libraries, which was because CentOS 8 only offers to install releases *35.0-1.el8, while the current installaion is on 32, and i did not want to risk complications from upgrading the package. I solved the problem by downloading the 32 release and installing them offline, which then satisfied the prerequisites. Afterwards the installation worked. I explain this "refusal" of EasyBuild to install the rdma-devel libraries to myself like EB found the rdma package preinstalled on OS level, and therefore also expects the relateed devel packages to be provided by the OS as well. Possibly uninstalling rdma on OS level would also have helped the situation. Best.

sjpb commented 2 years ago

Having just hit this I'd speculate that the reason EB can't install them is that they affect the entire system, i.e. they can't be isolated to a toolchain/package install (and also the rdma install etc will depend on the h/w, although I guess some sufficently-complex logic could work out what a good fit is).

akesandgren commented 2 years ago

The reason we can't install those using EB is that they depend on the corresponding kernel module. If you want OpenMPI (or nowadays UCX) to use Infiniband then you must have them installed from the OS. If you really don't have Infiniband you can either make a manual copy of the UCX easyconfig and remove the dependency, or write a hook, see the framework/contrib dir in your EB install) and remove them on the fly.

sjpb commented 2 years ago

Of course, thanks. Out of interest, should it still need those packages even on a non-RDMA/IB capable node? Presumably you can install foss on such machines?

akesandgren commented 2 years ago

You only need those packages and the dependency on systems which actually have IB hardware. Since the main target for EB is HPC sites we have that dependency there by default. But if you don't need it, write a hook to remove it on the fly and you won't have to remember to do it manually for every foss version...

boegel commented 2 years ago

Note that there's also --ignore-osdeps, which just lets EasyBuild skip the check for specified OS dependencies.

Obviously: use with care.