search

marvel-nccr / ansible-role-siesta

Ansible role for installing the siesta code
Other
0 stars 3 forks source link

⬆️ UPDATE: Siesta MaX-1.1 #12

Closed albgar closed 3 years ago

albgar commented 3 years ago

The updated role builds and deploys version 'MaX-1.1' of Siesta (with support for pseudopotentials in PSML format, and for Lua scripting, among other improvements). It does not build support for the ELSI library as this is more of a performance feature.

ltalirz commented 3 years ago

@chrisjsewell Feel free to merge this or to review as well

chrisjsewell commented 3 years ago

@albgar on a related matter that you may be able to help with, I've actually this week created a role for building abinit. In this I had an issue building for Ubuntu 20.04, because libblacs-mpi-dev was not available on apt (see apt installs here https://github.com/marvel-nccr/ansible-role-abinit/blob/master/tasks/main.yml, also link to docs https://www.netlib.org/blacs/). But in https://github.com/siesta-project/ansible-role-siesta/blob/master/tasks/main.yml I notice that you just skip installing it for ubuntu > 16. So is it just not required for these distros, or is there another package that takes its place?

albgar commented 3 years ago

@albgar on a related matter that you may be able to help with, I've actually this week created a role for building abinit. In this I had an issue building for Ubuntu 20.04, because libblacs-mpi-dev was not available on apt (see apt installs here https://github.com/marvel-nccr/ansible-role-abinit/blob/master/tasks/main.yml, also link to docs https://www.netlib.org/blacs/). But in https://github.com/siesta-project/ansible-role-siesta/blob/master/tasks/main.yml I notice that you just skip installing it for ubuntu > 16. So is it just not required for these distros, or is there another package that takes its place?

Newer versions of scalapack have the blacs "built-in". So you just need to install the scalapack-dev (I forgot the exact name) package.

chrisjsewell commented 3 years ago

thanks! ok I'll try that

albgar commented 3 years ago

@chrisjsewell: I have addressed all your comments (I think) and pushed, and the tests pass. But I do not really know how to tell the Github UI that I have done the checks requested...
I hope you can override this, or tell me what I need to do.

chrisjsewell commented 3 years ago

But I do not really know how to tell the Github UI that I have done the checks requested...

Thanks @albgar, I think what you are looking for is that you just click on the circle here, to request my re-review:

image

Also, in case you didn't know, you can use pre-commit install and this will ensure these tests pass before a commit is made (i.e. when you call git commit).

chrisjsewell commented 3 years ago

The updated role builds and deploys version 'MaX-1.1' of Siesta

Also one last thing, similar to what @ltalirz asked, with the changes to this role, what versions does it now support; only MaX-1.1 or is it e.g. back-compatible to v4.1, and what will potentially be involved to "upgrade" to the next version of siesta.

Also in terms of aiida-siesta, I assume the current version of that supports this version of siesta (and since what aiida-siesta version).

Perhaps some information like this could be added to the README.md, and also your comment about not supporting the ELSI library etc

albgar commented 3 years ago

We have currently a number of features in development waiting to be merged, and at the same time we are in the middle of a re-design of the building system for Siesta, so things are in flux. Hopefully, with the new building system in place and a more sedate version evolution, the role can inherit a bit more stability.

I have added a few more comments to README.md.

chrisjsewell commented 3 years ago

Thanks again @albgar but one last thing 😬

So when adding the new Abinit role to Quantum Mobile, I found that all the libxc tests were failing when using the apt installed libxc-dev, which installs v4.0.0 (see https://github.com/marvel-nccr/ansible-role-abinit/issues/9). But when I directly compiled v4.3.4 and linked to that it all worked. So I've created a seperate https://github.com/marvel-nccr/ansible-role-libxc role and in https://github.com/marvel-nccr/ansible-role-abinit/pull/10 allow for the direct specification of the path to libxc, e.g. pointing to /usr/local/libxc-4.3.4 rather than the apt installed /usr/local.

Given that Siesta also uses libxc; (a) will one of the default tests identify any issues with libxc, e.g. use a PAW xc-functional, and (b) if there are any issues, how easy/hard would it be to allow for a similar direct specification of the libxc path

edit: we could even consider moving tasks/libpsml.yml to this shared role, as I believe abinit can also link to that

albgar commented 3 years ago

This is strange... in the virtual machine I build on focal with my current role the version of libxc-dev is 4.3.4-1 (also stated in https://packages.ubuntu.com/search?keywords=libxc-dev&searchon=names&suite=focal&section=all), and not 4.0.0 as you seem to imply.

Nevertheless, the idea to install libxc explicitly is good, as we would have full control over it. However, I would like to re-use the Siesta role for things other than the vanilla Quantum Mobile, and then it would be better if the installation of libxc is local to the role, instead of assuming that somehow we are installing libxc earlier in the sequence of tasks. On the other hand, libxc installation can be time consuming, and the tests take long, so i can see the benefits of a "canonical" installation for everybody.

chrisjsewell commented 3 years ago

Yep that was the idea; to keep it modular. The abinit role does not intrinsically depend on the libxc one, it simply defaults to apt installing unless you set a predefined path. So this would not preclude the use of abinit or siesta outside of QM.

I note the link you provide only shows Ubuntu 20.04, perhaps for Ubuntu 18.04 (i.e. as currently used by QM) it is not as up-to-date?

chrisjsewell commented 3 years ago

Yep indeed it's a different version for 18.04: https://packages.ubuntu.com/search?keywords=libxc-dev&searchon=names&section=all

I do want to move QM to 20.04 eventually, but not quite there with all codes just yet!

albgar commented 3 years ago

I have implemented a mechanism to use the libxc role if the distribution's libxc-dev package is not up to standard. So in this sense I am using the libxc role as a "subroutine". However, a few things would need to be improved to make this more robust (some probably reflect my limited ansible skills):

Regarding the installation path, I see it as a welcome development that you are using sub-directories in /usr/local. Could everybody else do so? It would be good to have things in /usr/local/siesta-x.y.z, for example, in case we want to install different versions. Actually, support for modules (of the lmod kind) might be good too.

albgar commented 3 years ago

The CI checks fail because the libxc role cannot be found. How do I fix this?

chrisjsewell commented 3 years ago

If you have a look on the abinit role's molecule folder, you can see ho I do it there: look at the molecule.yml and requirements.yml

chrisjsewell commented 3 years ago

The libxc role does not currently install shared libraries. A client then needs to disable shared library support (e.g. in libgridxc) to use it. A role variable could be used to request shared libraries. However, this introduces more complications: what would happen if another role further down on the QM installation list actually installs a libxc-dev package, which has precedence in the shared libs search path?

Yes I must admit, I'm not an expert in the use of shared libraries. I did have a brief look before, if there was anything similar to python/conda virtual environments in the "apt world", but I didn't see anything obvious (although now that you mention lmod I guess that is somewhat similar). Any good reading/resources you could point me to would be welcome.

Regarding the installation path, I see it as a welcome development that you are using sub-directories in /usr/local. Could everybody else do so? It would be good to have things in /usr/local/siesta-x.y.z, for example, in case we want to install different versions. Actually, support for modules (of the lmod kind) might be good too.

Obviously the drawback is that the executables will not then be in the $PATH by default, and its just not something we have needed for now in Quantum Mobile, where we only have a single version of each code, and don't necessarily require the extra complication.

But yes I could see lmod as useful for the more general use of these roles.

Perhaps using something like https://github.com/idiv-biodiversity/ansible-role-lmod (this only supports redhat though) Then each role could write a ModuleFile and also, in some way, contribute to a meta module that just loads the "default" versions of all codes (a bit like how we have the release_notes role for generating a single file of all code versions).