search

EasyScience / EasyDiffractionApp

Diffraction data analysis application
https://easydiffraction.org
BSD 3-Clause "New" or "Revised" License
4 stars 1 forks source link

EXPLORE: Ways to package calculators statically #108

Closed rozyczko closed 3 years ago

rozyczko commented 3 years ago

We are bound to the python version used in compilation of CrysFML. This is too restrictive, we need to find a way to use or package calculators independently of the main app Python version

rozyczko commented 3 years ago

The most straightforward way (and the least work for us) is to have each calculator compiled as a static library, with python embedded. This isn't very easy, one needs to first compile Python statically on the build machine. BUT, there is an official conda-forge based distribution of static python libraries for all platform/version permutations. https://anaconda.org/conda-forge/libpython-static/files Note, that the archives can be downloaded without having to have Conda installed.

rozyczko commented 3 years ago

In case of CFML, one can build a static library, with python built in. This, however, is not immediately applicable, since in Python we can't import static libraries, only dynamic. This means we would have to write a C/C++ wrapper, dynamically compiled against CrysFML.a, which would forward declarations from the static library, such that the Python wrapper can use them. Using Cython seems the most straightforward, considering it is easy to create the wrapper. E.g. https://stackoverflow.com/questions/2105508/wrap-c-lib-with-cython

rozyczko commented 3 years ago

No Cython needed, just modify the existing makefile to include static python, similarly to how CFML.a is linked

/opt/intel/oneapi/compiler/2021.1.1/linux/bin/intel64/ifort -fPIC -warn -cpp  -qopt-report=0 -fPIC  -shared -nofor_main -Wl,-soname,crysfml_api.so -o crysfml_api.so
(.. fortran mod files ..)
../../Src/libcrysfml_common.a /home/piotr/anaconda3/lib/libpython3.8.so

and replace libpython3.8.so with libpython3.8.lib

rozyczko commented 3 years ago

Even better, deleting the dependence on libpython3.8.so is all that's required. The library compiles and has no dependency on python

$ ldd crysfml_api.so
        linux-vdso.so.1 (0x00007ffe93742000)
        libifport.so.5 => /opt/intel/oneapi/compiler/2021.1.1/linux/compiler/lib/intel64_lin/libifport.so.5 (0x00007f9d74dbd000)
        libifcoremt.so.5 => /opt/intel/oneapi/compiler/2021.1.1/linux/compiler/lib/intel64_lin/libifcoremt.so.5 (0x00007f9d75c3b000)
        libimf.so => /opt/intel/oneapi/compiler/2021.1.1/linux/compiler/lib/intel64_lin/libimf.so (0x00007f9d74735000)
        libsvml.so => /opt/intel/oneapi/compiler/2021.1.1/linux/compiler/lib/intel64_lin/libsvml.so (0x00007f9d72bbc000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9d7281e000)
        libintlc.so.5 => /opt/intel/oneapi/compiler/2021.1.1/linux/compiler/lib/intel64_lin/libintlc.so.5 (0x00007f9d725a6000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9d72387000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9d71f96000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9d71d7e000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9d71b7a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f9d75bb4000)

This seems to work just fine on with all python versions

rozyczko commented 3 years ago

Addressed by #114 and #115