Issue Installing PRESTO Python Packages in Python3.9 on RHEL 8

[JPGlaser]

Hey Scott,

Working on getting the PRESTO python packages all built up on our HPC node and noticed that for some reason (pulsar) [sysadmin@link presto]$ pip install . --user is giving the following error:

[...]
      /opt/python/3.9/envs/pulsar/compiler_compat/ld: /opt/pgplot/libpgplot.a(psdriv.o): relocation R_X86_64_32S against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
      /opt/python/3.9/envs/pulsar/compiler_compat/ld: /opt/pgplot/libpgplot.a(xwdriv.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
      /opt/python/3.9/envs/pulsar/compiler_compat/ld: /opt/pgplot/libpgplot.a(grclpl.o): relocation R_X86_64_32S against symbol `grcm00_' can not be used when making a shared object; recompile with -fPIC
      /opt/python/3.9/envs/pulsar/compiler_compat/ld: /opt/pgplot/libpgplot.a(grdot0.o): relocation R_X86_64_32S against symbol `grcm00_' can not be used when making a shared object; recompile with -fPIC
      /opt/python/3.9/envs/pulsar/compiler_compat/ld: /opt/pgplot/libpgplot.a(gritoc.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
      /opt/python/3.9/envs/pulsar/compiler_compat/ld: /opt/pgplot/libpgplot.a(grrec0.o): relocation R_X86_64_32S against symbol `grcm00_' can not be used when making a shared object; recompile with -fPIC
      /opt/python/3.9/envs/pulsar/compiler_compat/ld: /opt/pgplot/libpgplot.a(grtrml.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
      /opt/python/3.9/envs/pulsar/compiler_compat/ld: /opt/pgplot/libpgplot.a(pgcn01.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
      /opt/python/3.9/envs/pulsar/compiler_compat/ld: /opt/pgplot/libpgplot.a(grsyxd.o): warning: relocation against `grsymb_' in read-only section `.text'
      collect2: error: ld returned 1 exit status
      error: Command "gcc -pthread -B /opt/python/3.9/envs/pulsar/compiler_compat -shared -Wl,--allow-shlib-undefined -Wl,-rpath,/opt/python/3.9/envs/pulsar/lib -Wl,-rpath-link,/opt/python/3.9/envs/pulsar/lib -L/opt/python/3.9/envs/pulsar/lib -Wl,--allow-shlib-undefined -Wl,-rpath,/opt/python/3.9/envs/pulsar/lib -Wl,-rpath-link,/opt/python/3.9/envs/pulsar/lib -L/opt/python/3.9/envs/pulsar/lib build/temp.linux-x86_64-cpython-310/python/ppgplot_src/_ppgplot.o -L/usr/X11R6/lib -L/opt/pgplot -lcpgplot -lpgplot -lX11 -lpng -lm -o build/lib.linux-x86_64-cpython-310/_ppgplot.cpython-310-x86_64-linux-gnu.so -Wl,-rpath,/opt/pulsar/presto/lib" failed with exit status 1

########### EXT COMPILER OPTIMIZATION ###########
      Platform      :
        Architecture: x64
        Compiler    : gcc

      CPU baseline  :
        Requested   : 'min'
        Enabled     : SSE SSE2 SSE3
        Flags       : -msse -msse2 -msse3
        Extra checks: none

      CPU dispatch  :
        Requested   : 'max -xop -fma4'
        Enabled     : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_KNM AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
        Generated   : none
      CCompilerOpt.cache_flush[809] : write cache to path -> /opt/pulsar/presto/build/temp.linux-x86_64-cpython-310/ccompiler_opt_cache_ext.py
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for presto
Failed to build presto
ERROR: Could not build wheels for presto, which is required to install pyproject.toml-based projects

Any thoughts on what might be going wrong?

~ Joe G.

[scottransom]

Looks like the PGPLOT that is installed is statically compiled. I highly recommend using a pre-compiled PGPLOT package if at all possible. It definitely needs to be dynamically compliled. If you are doing the compiling yourself, I can probably help with the missing steps.

[JPGlaser]

So I did do a compile via Redhat, but that resulted in compile issues for PRESTO as it couldn't find where those directories were (because unlike Ubuntu, it scatters things rather than keeping them together).

Have you had any luck getting it compiled with the pgplot5 that RedHat8 has?

~ Joe G.

[scottransom]

No, I use pre-compiled packages on the various Linux machines that I use (Ubuntu for personal ones and RHEL at work, although it is RHEL6 and 7 there, and 8 is coming for us soon). But the RHEL packages that are available seem to work out of the box (and they have all the shared libraries in them). In fact, I just noticed that I've had my PGPLOT_DIR environment variable set incorrectly on my RHEL7 workstation and PRESTO is still working and finding everything! On my other, older, RHEL machine I have PGPLOT_DIR set to /usr/lib64/pgplot.

[JPGlaser]

So after talking with Nate, the conclusion is that if the following change is made to the Makefile, we can compile it with the pre-compiled packages for pgplot:

PGPLOTINC = -I/usr/include
PGPLOTLINK = -L/usr/lib64 -lcpgplot -lpgplot $(X11LINK) $(PNGLINK)

That gets everything linked correctly in RHEL8. ^_^

~ Joe G.

[scottransom]

That's good to know! So that's only a minor tweak from what is in the current Makefile.

[caseyjlaw]

I came across this issue (PRESTO on RHEL 8), as I found that the dnf package manager does not have some of the dependencies listed in the latest INSTALL.md file. The recommended command is:

apt install git build-essential libfftw3-bin libfftw3-dev pgplot5 libglib2.0-dev libcfitsio-bin libcfitsio-dev libpng-dev latex2html gfortran tcsh autoconf libx11-dev python3-dev python3-numpy python3-pip

Ones that are unavailable, but we'd rather not build are pgplot5 and fftw3. Are these generally available under these names for RHEL 8 (or even 7)?

[scottransom]

Hey Casey: This came up in another context, as well. So I have the answer ready to go.

NRAO uses RHEL / Centos, and my workstation is RHEL8. The PGPLOT rpms that NRAO's IT uses comes from here: https://rpmfusion.org/

fitzroy:~$ yum list | grep pgplot pgplot.x86_64 5.2.2-47.el8 @rpmfusion-nonfree-updates
pgplot-devel.x86_64 5.2.2-47.el8 @rpmfusion-nonfree-updates

For FFTW, we have:

fitzroy:~/Documents/cv$ yum list | grep fftw fftw.x86_64 3.3.5-11.el8 @rhel-8-for-x86_64-appstream-rpms
fftw-devel.x86_64 3.3.5-11.el8 @rhel-8-for-x86_64-appstream-rpms
fftw-libs.x86_64 3.3.5-11.el8 @rhel-8-for-x86_64-appstream-rpms
fftw-libs-double.x86_64 3.3.5-11.el8 @rhel-8-for-x86_64-appstream-rpms
fftw-libs-long.x86_64 3.3.5-11.el8 @rhel-8-for-x86_64-appstream-rpms
fftw-libs-quad.x86_64 3.3.5-11.el8 @rhel-8-for-x86_64-appstream-rpms
fftw-libs-single.x86_64 3.3.5-11.el8 @rhel-8-for-x86_64-appstream-rpms

And, just in case it helps, I also checked where the CFITSIO rpms come from, and that's here: https://docs.fedoraproject.org/en-US/epel/

fitzroy:~$ yum list | grep cfitsio cfitsio.x86_64 3.47-1.el8 @epel
cfitsio-devel.x86_64 3.47-1.el8 @epel

When I use those, I don't have to do anything special at all. It works exactly as on my home Ubuntu laptop (and other Ubuntu machines).

[caseyjlaw]

That's nice to hear. Thanks for supporting PRESTO so well.