ilastik

[boegel]

• link to support ticket: #2022112360000854
• website: https://www.ilastik.org
• installation docs: https://www.ilastik.org/documentation/basics/installation.html#basic-installation
• toolchain: ???
• easyblock to use: Binary?
• required dependencies:
  • ???
• optional dependencies:
  • [ ] CPLEX
  • [ ] Gurobi
• notes:
  • ...
• effort: (TBD)

[stevenvdb]

ilastik is provided as a self-contained archive (their run_ilastik.sh script cleans out LD_LIBRARY_PATH, so they really want you to use the libraries contained in the archive). I tested this on Hortense and both the GUI and CLI seem to work, so the easyconfig is currently based on the PackedBinary EasyBlock. If you think this approach is fine, I can open a PR in the central repository. They also provide source code, but getting all the dependencies right seems like a lot of work.

I have two questions:

• There is also a self-contained archive with GPU support. Is this something the user is interested in?
• When using the "learning tracking parameters" functionality, Gurobi is a required dependence. If this is something the user needs, a Gurobi library will need to be linked in the ilastik installation directory. I did not do this at the moment, as simply installing Gurobi already requires a license I think.

P.S. I have "Insufficient Rights" to access the support ticket on OTRS, so I can't ask the user directly.

[boegel]

There's no explicit mention of GPUs in the request, but if we can easily install it with GPU support, we can and let them check whether it's working.

The self-contained approach via run_ilastik.sh sounds easy, but it likely also results in a significantly slower installation that we could obtain by leveraging dependencies provided via EasyBuild. Especially for the GPU installation, this may be important (does the self-contained archive include CUDA?).

How "bad" is the list of required dependencies, can you compose an overview? Are they common dependencies, or also exotic/new stuff?

I guess we can start by installing the self-contained version, and see how well that works, if any problems arise, etc. So do make a PR for that, it's better than not having anything at all...

W.r.t. Gurobi: we can install without Gurobi for now, and see if that's sufficient. If the functionality enabled by Gurobi is really required, we can still look into that. What do you mean by "linked in the installation directory"? As symlink? Can we easily patch Ilastik to pick up Gurobi via $EBROOTGUROBI (if so, just loading the Gurobi module along with Ilastik could be enough)?

[stevenvdb]

There's no explicit mention of GPUs in the request, but if we can easily install it with GPU support, we can and let them check whether it's working.

The self-contained approach via run_ilastik.sh sounds easy, but it likely also results in a significantly slower installation that we could obtain by leveraging dependencies provided via EasyBuild. Especially for the GPU installation, this may be important (does the self-contained archive include CUDA?).

How likely is it to see a significant performance hit from the self-contained archive in your experience?

Yes, the self-contained archive for the GPU version includes CUDA.

How "bad" is the list of required dependencies, can you compose an overview? Are they common dependencies, or also exotic/new stuff?

This is the environment they use for their build: https://github.com/ilastik/ilastik/blob/main/conda-recipe/meta.yaml At first sight the only problematic one is that TensorFlow is fixed at version 1.14. Not sure if you want an old TensorFlow version with a newer toolchain in EasyBuild?

I guess we can start by installing the self-contained version, and see how well that works, if any problems arise, etc. So do make a PR for that, it's better than not having anything at all...

W.r.t. Gurobi: we can install without Gurobi for now, and see if that's sufficient. If the functionality enabled by Gurobi is really required, we can still look into that. What do you mean by "linked in the installation directory"? As symlink? Can we easily patch Ilastik to pick up Gurobi via $EBROOTGUROBI (if so, just loading the Gurobi module along with Ilastik could be enough)?

The provided archive contains a symlink to /opt/gurobi951/linux64/lib/libGurobiJni95.so. Before EasyBuild copies all files, that this symlink has to be removed (as the target of the link is not found). It is indeed possible to replace the symlink afterwards with ${EBROOTGUROBI}/lib/libGurobiJni95.so, then it should work if the Gurobi module is loaded together with ilastik.

[stevenvdb]

I am giving up on this one. I am just not convinced that putting a lot of effort into installing all dependencies ourselves is worth it. I would suggest the users start out with the self-contained installation that the developers of ilastik provide. If at some point they are not happy with the performance, we can pick it up again. Of course anybody is free to take over this issue if they want to have a go at it.