Problem with GLIBCXX_3.4.30 while importing package to python

[baanhle]

Hello, thank you for this amazing package. I've just installed PyOperon package with Nix Package Manager. But whenever i run the test file, i got these Error. ImportError: /nix/store/k2a429wpxgfwp4jaacl9iaqw4kxqjaxa-gcc-11.3.0-lib/lib/libstdc++.so.6: version 'GLIBCXX_3.4.30' not found (required by /nix/store/qry0r9qpw00n1knhhiq5gn0xlgzygyav-operon/lib/liboperon.so.0) One more problem about nix since i'm not familiar with it. After i close the terminal which i installed PyOperon and reopen another terminal, it seem like python cant find the package ModuleNotFoundError: No module named 'operon'

Best Regards

[foolnotion]

Hi,

The first issue is due to a mismatch between the glibc versions that python and operon are linked against, respectively. The solution to this is to LD_PRELOAD the right version. Simply run ldd and get the version of libstdc++ that operon links to:

ldd /nix/store/qry0r9qpw00n1knhhiq5gn0xlgzygyav-operon/lib/liboperon.so.0

The output should be something like this (the store paths will be different on your system):

$ ldd /nix/store/mlbng0chq7pixrq113jqlz59hi6f3x3l-operon/lib/liboperon.so.0
    linux-vdso.so.1 (0x00007fff07b9b000)
    libxxhash.so.0 => /nix/store/3a2kngdi6vhhl37hfqkqk3nb5bawmx7z-xxhash-0.8.1/lib/libxxhash.so.0 (0x00007f64dbfa0000)
    libfmt.so.8 => /nix/store/ys7pnszly9y7ry2basbflfc07cqrfixm-fmt-8.1.1/lib/libfmt.so.8 (0x00007f64dbf7c000)
    libopenlibm.so.4 => /nix/store/8d442rnf3p1agfa7q7si3m8yb1zg2jkl-openlibm-0.8.1/lib/libopenlibm.so.4 (0x00007f64dbf49000)
    libstdc++.so.6 => /nix/store/c2xvg0khan4cji4i84krnq2lwlsl4ab4-gcc-12.1.0-lib/lib/libstdc++.so.6 (0x00007f64dbc00000)
    libgcc_s.so.1 => /nix/store/7fz6dhhriwv6n4kdg05qi8cwf7mfqza9-glibc-2.35-163/lib/libgcc_s.so.1 (0x00007f64dbf2f000)
    libc.so.6 => /nix/store/7fz6dhhriwv6n4kdg05qi8cwf7mfqza9-glibc-2.35-163/lib/libc.so.6 (0x00007f64db800000)
    /nix/store/7fz6dhhriwv6n4kdg05qi8cwf7mfqza9-glibc-2.35-163/lib64/ld-linux-x86-64.so.2 (0x00007f64dc092000)
    libm.so.6 => /nix/store/7fz6dhhriwv6n4kdg05qi8cwf7mfqza9-glibc-2.35-163/lib/libm.so.6 (0x00007f64dbe4d000)

This is the line of interest:

libstdc++.so.6 => /nix/store/c2xvg0khan4cji4i84krnq2lwlsl4ab4-gcc-12.1.0-lib/lib/libstdc++.so.6 (0x00007f64dbc00000)

Then you do:

LD_PRELOAD=/nix/store/c2xvg0khan4cji4i84krnq2lwlsl4ab4-gcc-12.1.0-lib/lib/libstdc++.so.6 python example/operon-sklearn.py

(don't forget to use the actual paths from your system)

About the second issue, you actually need to be inside a nix develop shell for this to work. Otherwise the environment will not be set up for running operon.

[baanhle]

Hi, Thanks for the answer. I've done as you said ldd /nix/store/qry0r9qpw00n1knhhiq5gn0xlgzygyav-operon/lib/liboperon.so.0 Here is the output

        linux-vdso.so.1 (0x00007ffc84bea000)
        libxxhash.so.0 => /nix/store/3a2kngdi6vhhl37hfqkqk3nb5bawmx7z-xxhash-0.8.1/lib/libxxhash.so.0 (0x00007f9947459000)
        libfmt.so.7 => /nix/store/d61jypssq5f27ap9my3vxl33ky8ayrzv-fmt-7.1.3/lib/libfmt.so.7 (0x00007f994741e000)
        libopenlibm.so.4 => /nix/store/8d442rnf3p1agfa7q7si3m8yb1zg2jkl-openlibm-0.8.1/lib/libopenlibm.so.4 (0x00007f99473eb000)
        libstdc++.so.6 => /nix/store/c2xvg0khan4cji4i84krnq2lwlsl4ab4-gcc-12.1.0-lib/lib/libstdc++.so.6 (0x00007f99471c4000)
        libgcc_s.so.1 => /nix/store/7fz6dhhriwv6n4kdg05qi8cwf7mfqza9-glibc-2.35-163/lib/libgcc_s.so.1 (0x00007f99471aa000)
        libc.so.6 => /nix/store/7fz6dhhriwv6n4kdg05qi8cwf7mfqza9-glibc-2.35-163/lib/libc.so.6 (0x00007f9946f9e000)
        /nix/store/7fz6dhhriwv6n4kdg05qi8cwf7mfqza9-glibc-2.35-163/lib64/ld-linux-x86-64.so.2 (0x00007f994752d000)
        libm.so.6 => /nix/store/7fz6dhhriwv6n4kdg05qi8cwf7mfqza9-glibc-2.35-163/lib/libm.so.6 (0x00007f9946ebe000)

and then when i run LD_PRELOAD=/nix/store/c2xvg0khan4cji4i84krnq2lwlsl4ab4-gcc-12.1.0-lib/lib/libstdc++.so.6 python testop.py I get this error Illegal instruction

About the second issue, when when i run nix develop , it will start to redownload and fresh install from nothing.

[foolnotion]

Hi @baanhle, an illegal instruction typically means that the CPU was asked to execute an unsupported instruction or that the code you are running tries to do something illegal. Could you maybe share the code or specify which specific CPU model you're running on?

[baanhle]

Hi, my code have nothing but

import numpy as np
import operon

I'm running on WSL in Windows 10 with my CPU Intel Xeon E5-2689 v0 2.6Ghz

[foolnotion]

Sorry for the late reply, unfortunately there is no easy solution here. I believe that the underlying problem is that we try to provide a reasonable level of optimization when building operon/pyoperon. This is hardcoded in operon's flake.nix file:

"-DCMAKE_CXX_FLAGS=${if pkgs.targetPlatform.isx86_64 then "-march=haswell" else ""}"

I think the problem is that the specific Xeon model is not fully compatible with the haswell optimization level. I could not reproduce the illegal instruction issue locally.

One potential workaround is to git clone operon and pyoperon and change the optimization level in the flake file. The steps would be as follows:

1. clone operon somewhere locally let's name it <operon_dir>
2. go to <operon_dir> and edit flake.nix, change -march=haswell into something safer (e.g. x86_64)
3. clone pyoperon somewhere locally say <pyoperon_dir>

4. edit <pyoperon_dir>/flake.nix and make two changes:

   • line 9: change operon.url = "github:heal-research/operon/"; to operon.url = "path:<operon_dir>";. Note that the path must be absolute
   • line 61: change march=haswell to your desired optimization level
5. go to <pyoperon-dir> and run nix flake update
6. go to <pyoperon-dir> and run nix develop

Now you should obtain a dev shell just like before, with a more compatible optimization level. Another possibly safer alternative to haswell is to use the x86-64-v3 microarchitectural level (-march=x86-64-v3). Note that operon's performance is greatly influenced by the optimizations (at least avx/avx2 is recommended).

About the second issue, when when i run nix develop , it will start to redownload and fresh install from nothing.

Once the dev shell is cached, no redownload or recompilation should take place.

In the future hopefully we'll be able to make these flakes more user-friendly.

[baanhle]

Thanks for your answer but it doesn't work for me. Luckily, i manage to install it on another PC. I think we could close this issue.

[foolnotion]

Another option would be to use an install script such as this: https://github.com/cavalab/srbench/blob/Competition2022/official_competitors/operon/install.sh

This works well with conda. You'd still have to adjust compilation flags for your machine but it's more straightforward.

[noah-hoffmann]

Hi, I encounter a similar problem while trying to install pyoperon on Ubuntu 20.04 with pip. For installing I run: export LD_LIBRARY_PATH=/home/noah/miniconda3/envs/operon/lib/ pip install pyoperon If I then try to import pyoperon in python I get the following error: ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/noah/miniconda3/envs/operon/lib/python3.10/site-packages/pyoperon/pyoperon.cpython-310-x86_64-linux-gnu.so)

I already tried your @foolnotion approach and running LD_PRELOAD but this does not change anything.

[foolnotion]

hi @noah-hoffmann, thanks for reporting this,

it seems to be a pervasive issue unfortunately. possibly also related to #3

the python module is known to work on ubuntu 21.10 so my suspicion is that we have some backwards compatibility issues with older ubuntu/library versions. Likely the system libc version is too old, and we should generate a new pip release linked against the older version.

Could you list the output of the following two commands?

ldd <python site packages>/pyoperon/pyoperon.cpython-39-x86_64-linux-gnu.so

strings /lib/x86_64-linux-gnu/libc.so.6 | grep GLIBC

strings /lib/x86_64-linux-gnu/libstdc++.so.6 | grep CXXABI

[noah-hoffmann]

Thank you for your answer, the output of the commands are:

ldd ~/miniconda3/envs/operon/lib/python3.10/site-packages/pyoperon/pyoperon.cpython-310-x86_64-linux-gnu.so 
/bin/bash: /home/noah/miniconda3/envs/operon/lib/libtinfo.so.6: no version information available (required by /bin/bash)
/home/noah/miniconda3/envs/operon/lib/python3.10/site-packages/pyoperon/pyoperon.cpython-310-x86_64-linux-gnu.so: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/noah/miniconda3/envs/operon/lib/python3.10/site-packages/pyoperon/pyoperon.cpython-310-x86_64-linux-gnu.so)
/home/noah/miniconda3/envs/operon/lib/python3.10/site-packages/pyoperon/pyoperon.cpython-310-x86_64-linux-gnu.so: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /home/noah/miniconda3/envs/operon/lib/python3.10/site-packages/pyoperon/pyoperon.cpython-310-x86_64-linux-gnu.so)
    linux-vdso.so.1 (0x00007fffefd51000)
    /home/noah/miniconda3/envs/operon/lib/libstdc++.so.6 (0x00007fd2b0851000)
    libpython3.10.so.1.0 => /home/noah/miniconda3/envs/operon/lib/libpython3.10.so.1.0 (0x00007fd2b04ac000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd2b033c000)
    libgcc_s.so.1 => /home/noah/miniconda3/envs/operon/lib/libgcc_s.so.1 (0x00007fd2b0322000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd2b0130000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fd2b0c1e000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd2b010b000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd2b0105000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fd2b0100000)

strings /lib/x86_64-linux-gnu/libc.so.6 | grep GLIBC
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_2.13
GLIBC_2.14
GLIBC_2.15
GLIBC_2.16
GLIBC_2.17
GLIBC_2.18
GLIBC_2.22
GLIBC_2.23
GLIBC_2.24
GLIBC_2.25
GLIBC_2.26
GLIBC_2.27
GLIBC_2.28
GLIBC_2.29
GLIBC_2.30
GLIBC_PRIVATE
GNU C Library (Ubuntu GLIBC 2.31-0ubuntu9.9) stable release version 2.31.

Concerning your third command the file does not exist on my system.

[foolnotion]

hi @noah-hoffmann, thanks for the feedback. the problem is indeed an incompatibility in the glibc versions.

I've released pyoperon-0.3.3 which was built on ubuntu-18.04 (bionic) and should not have any more problems. I have already tested it in several ubuntu containers but I'd love to see that it works for you too.

please note that you will need the following support libraries:

conda install -c conda-forge xxhash fmt

full listing of ldd:

$ ldd build/lib.linux-x86_64-cpython-39/pyoperon/pyoperon.cpython-39-x86_64-linux-gnu.so 
    linux-vdso.so.1 (0x00007ffc9edec000)
    libpython3.9.so.1.0 => /home/ubuntu/mambaforge/envs/pyoperon-3.9/lib/libpython3.9.so.1.0 (0x00007f0999800000)
    libxxhash.so.0 => /home/ubuntu/mambaforge/envs/pyoperon-3.9/lib/libxxhash.so.0 (0x00007f0999e39000)
    libfmt.so.9 => /home/ubuntu/mambaforge/envs/pyoperon-3.9/lib/libfmt.so.9 (0x00007f0999e04000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0999400000)
    libstdc++.so.6 => /home/ubuntu/mambaforge/envs/pyoperon-3.9/lib/libstdc++.so.6 (0x00007f0999c50000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0999000000)
    libgcc_s.so.1 => /home/ubuntu/mambaforge/envs/pyoperon-3.9/lib/libgcc_s.so.1 (0x00007f0999c37000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0998c00000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f0999c00000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0998800000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f0998400000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f0998000000)

[noah-hoffmann]

Thank you @foolnotion, it seems to work now, although I had to specify the fmt version:

conda install -c conda-forge xxhash 'fmt>=9.0'