increase visibility of roles

[ltalirz]

We have all these nice ansible roles that know how to install various codes & tools and might be useful on their own. However, the roles are not easy to use on their own (few people are willing/able to learn ansible).

I think adding automated singularity builds for (some of) these roles would a big step as it makes them easy to use - and it's probably not that much work.

Step 1: For such "code roles"

• get rid of the become_user (see #67). still fine to keep become where it's needed (i.e. don't assume the role is run with become: true), but it will be needed a lot.

• let code_folder default to /tmp

• add prefix variable, which defaults to /usr/local

• [x] marvel-nccr.bigdft

• [x] marvel-nccr.cp2k

• [x] marvel-nccr.fleur

• [ ] marvel-nccr.gudhi

• [x] marvel-nccr.i_pi

• [x] marvel-nccr.lammps

• [ ] marvel-nccr.openbabel

• [ ] marvel-nccr.phtools

• [x] marvel-nccr.quantum_espresso

• [x] marvel-nccr.raspa

• [x] marvel-nccr.siesta

• [ ] marvel-nccr.v_sim

• [x] marvel-nccr.wannier90

• [x] marvel-nccr.yambo

• [x] marvel-nccr.zeopp

• [x] marvel-nccr.editors

• [x] marvel-nccr.slurm

Step 2: Start adding singularity recipes using the ansible roles

Start by adding a cookiecutter based on this recipe that creates a singularity recipe from an existing marvel-nccr role. Besides the role names, I expect very little difference between these singularity recipes. Would be nice if a new release of the ansible role could re-trigger the build of the singularity image.

• [ ] marvel-nccr.bigdft
• [ ] marvel-nccr.cp2k
• [ ] marvel-nccr.fleur
• [ ] marvel-nccr.gudhi
• [ ] marvel-nccr.i_pi
• [ ] marvel-nccr.lammps
• [ ] marvel-nccr.openbabel
• [ ] marvel-nccr.phtools
• [ ] marvel-nccr.quantum_espresso
• [ ] marvel-nccr.raspa
• [ ] marvel-nccr.siesta
• [ ] marvel-nccr.v_sim
• [ ] marvel-nccr.wannier90
• [ ] marvel-nccr.yambo
• [ ] marvel-nccr.zeopp
• [ ] marvel-nccr.editors
• [ ] marvel-nccr.slurm

Eventually, this would also lead to the benefit that one could release a "minimal" version of Quantum Mobile that does not include any of the above codes but can easily run them through singularity

[ltalirz]

Note: One advantage we had from being able to install QE in a separate folder was the the singularity recipe could build everything in /tmp directly on the "host filesystem" and then only copy the executables over to the actual image (without having to worry about cleaning up temporary files). It might make sense to retain this feature (which should be the case with a configurable prefix, however).

[ltalirz]

After realising that SingularityHub currently blocks IP addresses after 10 pulls per day (which can easily create issues when using singularity files in CI tests), I have decided to stop moving in the singularity direction for the time being and reconsider docker / DockerHub instead due to its maturity.

I've also learned that you can use docker multistage builds (Dockerfile.build) to e.g. copy the executables from a range of "code" images (e.g. QE, siesta, ...) to build one container that contains all of them (=> Quantum Mobile). One could ask code maintainers to provide:

• All binaries & libraries in a given location, e.g. /usr/local
• (optional) A module file to set some environment variables

[ltalirz]

While one could still build docker images from the ansible roles, the underlying purpose of this issue (make the codes more easily available to users without Quantum Mobile) will in my view be better achieved via the conda route, with its fully-fledged dependency management that is indispensable as soon as you start installing several codes side-by-side.

I'm closing this.

[chrisjsewell]

dependency management that is indispensable as soon as you start installing several codes side-by-side

@ltalirz FYI I don't think you would be able to install the codes into the same conda environment; you would create a separate one for each code. I say this because I've already tried creating an environment with multiple MaX codes and there is a lot of dependency incompatibilities

[ltalirz]

@ltalirz FYI I don't think you would be able to install the codes into the same conda environment; you would create a separate one for each code. I say this because I've already tried creating an environment with multiple MaX codes and there is a lot of dependency incompatibilities

This is one of the reasons why I think it would be important to get the developers (of the roles) to sit together. In some cases, incompatible dependencies are unavoidable, and then it's fine, but in many cases it can also be avoided and we should also try to maximize compatibility in the MaX conda-forge roles (where it makes sense).

Example: all codes should try to support openmpi as the MPI library (other libraries optional of course)

On the upside, conda will reuse (hardlink) dependencies when they are the same, even if the codes are all installed in separate environments.

P.S. I'm sure similar discussions have already been had in the context of the ESL, i.e. it would be good to learn from the insights they collected.

[chrisjsewell]

On the upside, conda will reuse (hardlink) dependencies when they are the same, even if the codes are all installed in separate environments.

Exactly, that's why I don't think it is too much of a problem. It may be possible, but a lot more work to keep ~10 or more codes "in sync" with no version incompatibilities.

[ltalirz]

Perhaps dump the incompatibilities you encountered in https://github.com/marvel-nccr/quantum-mobile/issues/161