Use Hermes for the fastest io

[tomsail]

One would like to benefit from the fastest library available to load selafin files: TelemacFile (via HERMES)

There are two practical requisites to build this xarray package:

1. Have a minimal working setup (the least required libraries possible) -- Ease of installation
2. Use the fastest tools available - Performance

Point 2. requires the hermes libraries to be recognised in the python environment.

I reproduced a minimal setup on a new branch hermes.

Without the HERMES library:

$ python tests/perf_test.py 
Warning: Using SerafinFile. It is recommended to compile Hermes api
Time taken by telemac: 0.049854278564453125 seconds
Time taken by selafin: 0.007260560989379883 seconds

to be able to have the library working, I needed first to see which libraries I needed:

$ ldd _hermes.cpython-311-x86_64-linux-gnu.so 
        linux-vdso.so.1 (0x00007ffe06530000)
        libhermes4api.so => not found
        libspecial4api.so => not found
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8275c00000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f8275ebd000)

I had to load my telemac conda environment by doing:

mamba activate telemac 
mamba activate --stack slf 

slf being the environment where I have the package

and all libraries were then recognised in the path

$ ldd _hermes.cpython-311-x86_64-linux-gnu.so 
        linux-vdso.so.1 (0x00007ffc27f09000)
        libhermes4api.so => /home/tomsail/miniconda3/envs/telemac/opentelemac/builds/gnu.dynamic/wrap_api/lib/libhermes4api.so (0x00007ff6e2e00000)
        libspecial4api.so => /home/tomsail/miniconda3/envs/telemac/opentelemac/builds/gnu.dynamic/wrap_api/lib/libspecial4api.so (0x00007ff6e35c3000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff6e2a00000)
        libmpi_usempif08.so.40 => /home/tomsail/miniconda3/envs/telemac/lib/libmpi_usempif08.so.40 (0x00007ff6e356b000)
        libmpi_usempi_ignore_tkr.so.40 => /home/tomsail/miniconda3/envs/telemac/lib/libmpi_usempi_ignore_tkr.so.40 (0x00007ff6e355b000)
        libmpi_mpifh.so.40 => /home/tomsail/miniconda3/envs/telemac/lib/libmpi_mpifh.so.40 (0x00007ff6e2d96000)
        libmpi.so.40 => /home/tomsail/miniconda3/envs/telemac/lib/libmpi.so.40 (0x00007ff6e2c74000)
        libgfortran.so.5 => /home/tomsail/miniconda3/envs/telemac/lib/libgfortran.so.5 (0x00007ff6e2855000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff6e276e000)
        libgcc_s.so.1 => /home/tomsail/miniconda3/envs/telemac/lib/libgcc_s.so.1 (0x00007ff6e2c59000)
        libquadmath.so.0 => /home/tomsail/miniconda3/envs/telemac/lib/libquadmath.so.0 (0x00007ff6e2735000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff6e2c54000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff6e35f1000)
        libopen-pal.so.40 => /home/tomsail/miniconda3/envs/telemac/lib/./libopen-pal.so.40 (0x00007ff6e2635000)
        libopen-rte.so.40 => /home/tomsail/miniconda3/envs/telemac/lib/./libopen-rte.so.40 (0x00007ff6e257b000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007ff6e2c4f000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff6e2c4a000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007ff6e2c45000)
        libz.so.1 => /home/tomsail/miniconda3/envs/telemac/lib/././libz.so.1 (0x00007ff6e2c2a000)

$ python tests/perf_test.py 
Time taken by telemac: 0.00033593177795410156 seconds
Time taken by selafin: 0.007600069046020508 seconds

@pmav99, the good news: it appears that the hermes libraries are correctly installed on my conda (I double checked it) the tricky one is that I don't know how to make this environment minimal now..

A hint maybe for CI builds could be the matrix build I have set up for the conda environment.

ping @nicogodet, I think this might interesting for you when we figure out something minimal, so you can adapt it for windows. Let me know also if you have any idea to make _hermes more portable

[nicogodet]

A static build might be more portable but not 100% sure about this.

You can also reduce the number of built module in systel.cfg and option: hermes_only (as api requires all modules).

FYI, this is on my todo list to integrate it in QGIS processing script (very low priority)

[pmav99]

If it is tricky to build, then perhaps we should just try to recognize if HERMES is available or not. If it is we use it, if not we fall back to the Selafin class (or whatever is the fastest alternative).

Supporting 2 "modes" means that a bit more work is needed on the backend side, but chances are that it is less work compared to trying to build/test wheels etc.

[tomsail]

@nicogodet yes this could be quickest to build it in CI. What I fear is that _hermes cascades down to fortran and mpi libraries that could conflict with HPC native libraries (as you can see in the trace above). (I have experienced those conflicts with the opentelemac conda library already)

@pmav99

Supporting 2 "modes" means that a bit more work is needed on the backend side, but chances are that it is less work compared to trying to build/test wheels etc.

this is what I was thinking too.

[nicogodet]

Using gnu.shared and gnu.static from https://gitlab.pam-retd.fr/otm/telemac-mascaret/-/blob/main/configs/systel.debian.cfg

shared:

$ ldd _hermes.cpython-38-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007ffdf4966000)
        libhermes4api.so (0x00007f5560dc7000)
        libspecial4api.so (0x00007f5560dba000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5560bbb000)
        libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007f55608f3000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f55608d0000)
        libmpi_mpifh.so.40 => /lib/x86_64-linux-gnu/libmpi_mpifh.so.40 (0x00007f5560869000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f556153f000)
        libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f556081f000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f55606d0000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f55606b5000)
        libmpi.so.40 => /lib/x86_64-linux-gnu/libmpi.so.40 (0x00007f5560590000)
        libopen-pal.so.40 => /lib/x86_64-linux-gnu/libopen-pal.so.40 (0x00007f55604e2000)
        libopen-rte.so.40 => /lib/x86_64-linux-gnu/libopen-rte.so.40 (0x00007f5560426000)
        libhwloc.so.15 => /lib/x86_64-linux-gnu/libhwloc.so.15 (0x00007f55603d5000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f55603cf000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f55603ca000)
        libevent-2.1.so.7 => /lib/x86_64-linux-gnu/libevent-2.1.so.7 (0x00007f5560374000)
        libevent_pthreads-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_pthreads-2.1.so.7 (0x00007f556036f000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f5560351000)
        libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007f5560324000)
        libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f5560319000)

static:

$ ldd _hermes.cpython-38-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007fff71f40000)
        libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007f08fd85e000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f08fd66c000)
        libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f08fd622000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f08fd4d3000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f08fd4b8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f08fe2aa000)

Using a static link for fortran and gcc, additionnal f2py options are required at compile time but it seems not possible to add them from systel.cfg

[tomsail]

We switched to pyTelTools and don't need HERMES library anymore more info directly in the thread of the forum