search

vscentrum / vsc-software-stack

Central repository of easyconfigs used in the software installations on VSC clusters.
2 stars 6 forks source link

Add iCommands easyconfig and its dependencies #244

Closed stevenvdb closed 5 months ago

stevenvdb commented 1 year ago

My colleagues asked to provide an installation of iCommands that can be done with EasyBuild, to make it available on Hortense. @boegel Can you review the provided easyconfigs and/or install them on Hortense? For now I requested merging into site-kul, but maybe the vsc branch is more appropriate if it is ready for Hortense?

Notes:

laraPPr commented 10 months ago

@stevenvdb Can you open a ticket here so that we have all the information on the installation request?

laraPPr commented 10 months ago

It is also best to open a PR in https://github.com/easybuilders/easybuild-easyconfigs if you want it installed on hortense and reviewed by @boegel.

boegel commented 10 months ago

There's no real need for a software installation request, but a PR to the easyconfigs repo so we can get these included in EasyBuild would indeed be nice (unless there's a reason we can't add these easyconfigs to EasyBuild)

stevenvdb commented 10 months ago

There's no real need for a software installation request, but a PR to the easyconfigs repo so we can get these included in EasyBuild would indeed be nice (unless there's a reason we can't add these easyconfigs to EasyBuild)

Thanks for the feedback. My motivation for opening a PR here and not in the easyconfigs repo, was that I thought the threshold to get things merge would be lower here. I'll open another PR in the official repo, taking into account your comments.

One remark; I tried to stick with the dependency versions that you get when you install iRODS from an rpm. This indeed leads to situations where easyconfigs are introduced with only a (small) version difference compared to existing easyconfigs. I'll try to reuse existing versions where possible.

boegel commented 10 months ago

@stevenvdb Don't let my review stop a merge in the site-kul branch, what gets merged in there is up to the KUL team, not me.

But to get iCommands installed on Hortense, we strongly prefer having it in sync with EasyBuild as much as possible.

We briefly discussed this during the VSC CUE meeting today BTW. Is there a particular reason you prefer installing this with EasyBuild, rather than via an RPM?

cc @lexming

stevenvdb commented 10 months ago

We briefly discussed this during the VSC CUE meeting today BTW. Is there a particular reason you prefer installing this with EasyBuild, rather than via an RPM?

Not at all; the reason for my attempt to do this with EasyBuild, was that I thought you preferred that over an RPM. If the other VSC sites are willing to do a system-wide installation, there is no real need for these easyconfigs. On our clusters we install iCommands from an RPM, specifically version 4.3.1 (see https://packages.irods.org/yum/pool/el8/x86_64/irods-icommands-4.3.1-0.el8.x86_64.rpm and https://irods.org/download/)

stevenvdb commented 5 months ago

I will be closing this, since in the end iCommands were installed at the different sites with an RPM.

If anybody ever wants to pick this up and create an easyconfig, here is my experience; the files currently in this PR can certainly improve, for instance by implementing the suggestions from @boegel However, I don't think a clean EasyBuild installation will be possible, not because of EasyBuild limitations but rather because of the way iCommands/iRODS is built. Here are some interesting (recent) quotes from iRODS developers:

- iRODS and most of our externals are built with clang
- Much of our code is not gcc-friendly

I attempted to use the GCC toolchain, but quickly got stuck. This is why in my PR I used clang instead. Even though theoretically you should be able to use libraries compiled with gcc, there are a few cases such as Boost where ABI incompatibilities arise. So you need to compile Boost with clang, which then uncovers additional bugs that require new patches. In the end, you end up with quite a lot of custom easyconfigs and patches for the dependencies (note that the iRODS RPM ships with a number of dependencies that are kept separated from other system libraries, check /opt/irods-externals), which is not easy to maintain.

So my conclusion is that an RPM installation of iCommands/iRODS is preferred over a compilation from source. Perhaps if the iRODS build system significantly improves in the future, I'll change my opinion.