Provide easyconfig files for EasyBuild

[awvwgk]

I'm opening this issue to coordinate building xtb with EasyBuild, this issue came up originally at https://github.com/grimme-lab/xtb/issues/131#issuecomment-595972785.

Resources for EasyBuild are found at their read-the-docs page: https://easybuild.readthedocs.io/en/latest/

Ideally the easyconfig files would be available in this repository or as a submodule in the assets directory similar to the AUR PKGBUILD files.

Thanks to @dplacencia for bringing this to my attention.

[dplacencia]

Hi @awvwgk , thanks for open this issue. Like I said to you, I test newly with GCC 8.3.0 thas is part of the EasyBuild toolchain foss-2019a. I have a new problem. Meson is looking for 'lapack' library in meson.build script, and "Fortran library 'lapack' not found" error is jumping because foss-2019a use OpenBLAS for blas/lapack libraries. Presumably I think to remove line 99 of meson.build file "dependencies += fc.find_library('lapack', required: true)" could be work or maybe use 'custom_libraries' option. I'm not sure what's the best choice. What do you think about it?

[awvwgk]

Let's try it, I just dropped https://github.com/grimme-lab/xtb/blob/2362cd3334b385e6efb64360939d986f66e2e949/meson.build#L99 from my meson.build and run a build with GCC 8.3.0

CC=gcc-8 FC=gfortran-8 meson setup build_openblas -Dla_backend=openblas --buildtype release --optimization 2 --warnlevel 0

The link line fails with unresolved LAPACK dependencies, because my OpenBLAS package only provides a BLAS interface, but no LAPACK interface: https://www.archlinux.org/packages/community/x86_64/openblas/.

I think it is not save to assume OpenBLAS is providing LAPACK as well, in case your OpenBLAS version does and you know what you are doing, go for the custom_libraries option.

[dplacencia]

Hello again. I am trying to build xtb but this time with the arguments

-Dla_backend='custom' -Dcustom_libraries='openblas' --optimization=2 --buildtype release --warnlevel 0

I am getting two errors:

============== ERRORS =========================
______________ ERROR collecting test/test_interface.py _______
ImportError while importing test module '/opt/apps/easybuild/build/xTB/6.3.0/foss-2019a/xtb-master/python/xtb/test/test_interface.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../xtb-master/python/xtb/test/test_interface.py:48: in <module>
    class Testing:
../xtb-master/python/xtb/test/test_interface.py:51: in Testing
    from numpy import array
E   ModuleNotFoundError: No module named 'numpy'
___________ ERROR collecting test/test_solvation.py _______
ImportError while importing test module '/opt/apps/easybuild/build/xTB/6.3.0/foss-2019a/xtb-master/python/xtb/test/test_solvation.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../xtb-master/python/xtb/test/test_solvation.py:20: in <module>
    class Testing:
../xtb-master/python/xtb/test/test_solvation.py:21: in Testing
    from ase import Atoms
E   ModuleNotFoundError: No module named 'ase'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection 

These Python modules that give that error should I have them previously installed on the system or is an error inherent in xtb?

[dplacencia]

Another question. Where can I download the CREST source code? Or is it only in binary version?

[awvwgk]

If you only need the executable and the C-API than this warning does not matter to you. If you also want to support the Python-API and the ASE-Wrapper than you need ctypes and numpy for the bindings from Python to the C-API.

crest is for now only available as binary version and distributed along with xtb.

[dplacencia]

Hi, I just saw you commented on the xtb pull request on EasyBuild (https://github.com/easybuilders/easybuild-easyconfigs/pull/9993), thanks for your valuable help. Please ask me to visit the CREST pull request and clarify some doubts that we have as well. It is located at: https://github.com/easybuilders/easybuild-easyconfigs/pull/10040 Thanks

[dplacencia]

If you only need the executable and the C-API than this warning does not matter to you. If you also want to support the Python-API and the ASE-Wrapper than you need ctypes and numpy for the bindings from Python to the C-API.

This was not clear to me. Compilation stops when these two errors are thrown. If I want only the executable how can I skip these dependencies?

[awvwgk]

Sorry, why do you repost your previous comment in Spanish?

[dplacencia]

Sorry, why do you repost your previous comment in Spanish?

I'm so sorry, I didn't notice writing it in English. A little cross between projects, sorry

[awvwgk]

This was not clear to me. Compilation stops when these two errors are thrown. If I want only the executable how can I skip these dependencies?

The compilation should be finished if you encounter this type of error, as it happens in the testsuite of the (optional) Python wrapper. Currently the test for the Python wrapper are also managed in the meson.build file (maybe I should change this) and automatically enabled if the right tool is installed (pytest, in this case). Right now the Python wrapper can't be enabled or disabled explicitly (I should change this as well).

Still, this does not affect the integrity of the xtb binary which is already present.

[dplacencia]

So, if i remove the

pytest = find_program('pytest', required: false)
if pytest.found()
  test('pytest: xtb.py', pytest, args: ['--pyargs', 'xtb'], env: xtbenv)
endif

lines from the meson.build file should it not influence the compilation of the xtb binary?

[awvwgk]

137 should fix the Python issue by disabling the testing code with -Dpython=false.

[dplacencia]

Hello, in the build step I am getting these fails: Ok: 49 Expected Fail: 5 Fail: 0 Unexpected Pass: 0 Skipped: 3 Timeout: 5

The tests that fail due to timeout are: 5/62 Singlepoint TIMEOUT 30.05 s 6/62 Geometry opt. TIMEOUT 30.04 s 7/62 IP/EA TIMEOUT 30.08 s 9/62 GFN1-xTB TIMEOUT 30.03 s 10/62 GFN2-xTB/GBSA TIMEOUT 30.04 s

at the end of each test appears: ######################################################################## [WARNING] Please study the warnings concerning your input carefully -3- userdata_read: could not find 'xcontrol' -2- set_rdcontrol: could not find 'xcontrol' -1- prog_main: XTBHOME is not set, using HOME instead ########################################################################

Is there any argument that I can pass to avoid this timeout?

[awvwgk]

Sorry, this is not the right strategy, those are integrity tests for the production binary and should not time out with a proper linear algebra backend, if they do your compilation can hardly handle any real computations (this are only 24 atom systems!).

The longest of those test (6 Geometry opt.) is only taking two seconds on my laptop (Intel i7-3520M CPU @ 2.90GHz) with a GCC 8.3.0 compilation and MKL backend.

[awvwgk]

Also, you can use three backticks (```) in GitHub flavored markdown to format code blocks, see https://help.github.com/en/github/writing-on-github/basic-writing-and-formatting-syntax

[awvwgk]

After looking more detailed into this, I found that I disabled the optimization test for reference LAPACK/BLAS on the CI server due to time out problems. This is okay for testing a commit, but not for deploying a production build.

This also means, your current setup is not faster than reference LAPACK/BLAS, which is not acceptable. Anyway, would you mind sharing your easyconfig file so I don't have to keep guessing what's wrong with your setup?

[dplacencia]

I just made a commit of my current configuration #9993, please review it to try to find a solution.

[dplacencia]

Well, finally this combination was successful for me xtb-6.2.3-foss-2019a.eb Soon I will do some tests with xtb to verify that it works well. The next step I think would be adding xtb release 6.2.3

[awvwgk]

The release 6.2.3 was staged already and is tested internally by us right now. It might take another week until we have iterated over it for the next stable version.

[dplacencia]

Okay, I'll be waiting for the new release to send the merge request from the EasyBuild master branch. I am enormously grateful for your help

[awvwgk]

@dplacencia The release it out (https://github.com/grimme-lab/xtb/releases/tag/v6.2.3), you have a day or two to get your easyconfig merged before we post the announcement. If it is merged in time we will mention it in the announcement.

[dplacencia]

Thanks a lot. I will do my best to merge it out on time. Regards

[dplacencia]

@awvwgk , is there any news from @pprcht ? Please put him in contact with me to discuss the permission of CREST publication in EasyBuild.

[awvwgk]

@dplacencia you want something from us, so just write an email to our mailing list.

[dplacencia]

I couldn't find the mailing list. Could you tell me the address? sorry

[awvwgk]

xtb@thch.uni-bonn.de at the top of https://github.com/grimme-lab