[BUG] ImportError of GLIBCXX in Catalyst frontend

glassnotes commented 1 year ago

Issue description

I am experiencing issues when importing Catalyst that seem to depend on where and how the library is imported. It might be specific to my installation (I had to hack around a bit to get it to pick up clang), but I thought I should report because the failure seems erratic. I think it is related to something in the frontend.

Expected behavior: (What you expect to happen)

Importing Catalyst from both shell, interpreter, and Jupyter notebook should work without errors.

Actual behavior: (What actually happens)

When I fire up an interpreter, I can do

>>> from catalyst import qjit

or

>>> import catalyst

without issue.

If I have a Python script, and begin it with

from catalyst import qjit

it works when this is the only line in the script, but in other contexts (my Shor implementation), I receive the following Traceback:

Traceback (most recent call last):
  File "/home/olivia/Code/shortalyst-dev/full_jit_working.py", line 10, in <module>
    from catalyst import cond, measure, qjit, for_loop, while_loop
  File "/home/olivia/Code/catalyst/frontend/catalyst/__init__.py", line 64, in <module>
    from catalyst.compilation_pipelines import QJIT, CompileOptions, qjit
  File "/home/olivia/Code/catalyst/frontend/catalyst/compilation_pipelines.py", line 40, in <module>
    from catalyst.compiler import CompileOptions, Compiler
  File "/home/olivia/Code/catalyst/frontend/catalyst/compiler.py", line 30, in <module>
    from mlir_quantum.compiler_driver import run_compiler_driver
ImportError: /home/olivia/.conda/envs/catalyst/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/olivia/Code/catalyst/frontend/catalyst/../../mlir/build/python_packages/quantum/mlir_quantum/compiler_driver.so)

However, adding a line above this import that does import catalyst solves the problem

In a Jupyter Notebook, neither version works, and I get a similar but more detailed error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[2], line 1
----> 1 import catalyst
      3 from catalyst import qjit

File ~/Code/catalyst/frontend/catalyst/__init__.py:64
     58 sys.modules["mlir_quantum._mlir_libs._quantumDialects.quantum"] = types.ModuleType(
     59     "mlir_quantum._mlir_libs._quantumDialects.quantum"
     60 )
     63 from catalyst.ag_utils import AutoGraphError, autograph_source
---> 64 from catalyst.compilation_pipelines import QJIT, CompileOptions, qjit
     65 from catalyst.pennylane_extensions import (
     66     adjoint,
     67     cond,
   (...)
     74     while_loop,
     75 )
     76 from catalyst.utils.exceptions import CompileError

File ~/Code/catalyst/frontend/catalyst/compilation_pipelines.py:40
     38 import catalyst
     39 from catalyst.ag_utils import run_autograph
---> 40 from catalyst.compiler import CompileOptions, Compiler
     41 from catalyst.jax_tracer import trace_to_mlir
     42 from catalyst.pennylane_extensions import QFunc

File ~/Code/catalyst/frontend/catalyst/compiler.py:30
     27 from io import TextIOWrapper
     28 from typing import Any, List, Optional
---> 30 from mlir_quantum.compiler_driver import run_compiler_driver
     32 from catalyst._configuration import INSTALLED
     33 from catalyst.utils.exceptions import CompileError

ImportError: /home/olivia/.conda/envs/catalyst/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/olivia/Code/catalyst/frontend/catalyst/../../mlir/build/python_packages/quantum/mlir_quantum/compiler_driver.so)

Reproduces how often: (What percentage of the time does it reproduce?)

Erratic (see description above)

System information: (post the output of import pennylane as qml; qml.about())

Running on Ubuntu 22.04.

Catalyst is installed from source off of main branch commit 422dc14879071039a9a64eb781627b6b441174de.

Clang++ is version 14.0.0-1ubuntu1.1.

Output of qml.about() (packages installed from Catalyst's requirements.txt file):

Name: PennyLane
Version: 0.32.0
Summary: PennyLane is a Python quantum machine learning library by Xanadu Inc.
Home-page: https://github.com/PennyLaneAI/pennylane
Author: 
Author-email: 
License: Apache License 2.0
Location: /home/olivia/.conda/envs/catalyst/lib/python3.11/site-packages
Requires: appdirs, autograd, autoray, cachetools, networkx, numpy, pennylane-lightning, requests, rustworkx, scipy, semantic-version, toml, typing-extensions
Required-by: pennylane-catalyst, PennyLane-Lightning

Platform info:           Linux-6.2.0-33-generic-x86_64-with-glibc2.35
Python version:          3.11.0
Numpy version:           1.23.5
Scipy version:           1.10.0
Installed devices:
- default.gaussian (PennyLane-0.32.0)
- default.mixed (PennyLane-0.32.0)
- default.qubit (PennyLane-0.32.0)
- default.qubit.autograd (PennyLane-0.32.0)
- default.qubit.jax (PennyLane-0.32.0)
- default.qubit.tf (PennyLane-0.32.0)
- default.qubit.torch (PennyLane-0.32.0)
- default.qutrit (PennyLane-0.32.0)
- null.qubit (PennyLane-0.32.0)
- lightning.qubit (PennyLane-Lightning-0.32.0)

Source code and tracebacks

N/A

Additional information

Let me know what else would be useful!

erick-xanadu commented 1 year ago

Thanks for reporting this error @glassnotes! It looks to be related to the new compiler driver integration.

glassnotes commented 1 year ago

Thanks for the quick reply, @erick-xanadu !

Some more info for you: upon further testing, in a script, if I import packages in the following order,

import catalyst
import jax.numpy as jnp

I do not get an error, however if I reverse the two lines, I get the error listed above. I've just updated both jax and jaxlib to be versions 0.4.14.

maliasadi commented 1 year ago

Hi @glassnotes, Thanks for reporting this issue.

This comes from the fact that you have multiple versions of libstdc++ on your machine. The compiler_driver library of Catalyst was apparently built with a different version of this library. You can find it by running ldd /home/olivia/Code/catalyst/frontend/catalyst/../../mlir/build/python_packages/quantum/mlir_quantum/compiler_driver.so. The path to libstdc++.so should be something like /lib/x86_64-linux-gnu/libstdc++.so.6.

The conda version of libstdc++ doesn't include GLIBCXX_3.4.30 while the gcc package (in /lib/libstdc++.so.6) has it! The compiler_driver is linked against the gcc package but it doesn't find the lib when it's called from a conda venv.

To fix this issue, you can replace the conda version of this library by this one,

cp /lib/x86_64-linux-gnu/libstdc++.so.6 /home/olivia/.conda/envs/catalyst/bin/../lib/libstdc++.so.6

Please let us know if this also fixes the issue or you face any other problems with installing Catalyst.

dime10 commented 1 year ago

The question is, how do we prevent this from happening more generally? 🤔

glassnotes commented 1 year ago

Copying the library in does work, but feels a little sketchy :sweat_smile:

I ended up with multiple versions of the library during the installation process. In particular, despite having clang installed on my machine, I initially received the output

CMake Error at /home/olivia/.conda/envs/catalyst/lib/python3.11/site-packages/cmake/data/share/cmake-3.27/Modules/CMakeTestCXXCompiler.cmake:60 (message):
  The C++ compiler

    "/usr/bin/clang++"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: '/home/olivia/Code/catalyst/runtime/build/CMakeFiles/CMakeScratch/TryCompile-k56B71'

    Run Build Command(s): /usr/bin/ninja -v cmTC_54cbb
    [1/2] /usr/bin/clang++    -MD -MT CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -MF CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o.d -o CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -c /home/olivia/Code/catalyst/runtime/build/CMakeFiles/CMakeScratch/TryCompile-k56B71/testCXXCompiler.cxx
    [2/2] : && /usr/bin/clang++   CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -o cmTC_54cbb   && :
    FAILED: cmTC_54cbb 
    : && /usr/bin/clang++   CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -o cmTC_54cbb   && :
    /usr/bin/ld: cannot find -lstdc++: No such file or directory
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    ninja: build stopped: subcommand failed.

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:3 (project)

when trying to run make all in the top-level Catalyst directory. This led me to installing libstdc++-12-dev, which fixed the previous problem, but resulted in the new one in the issue description.

For what it's worth, the "not being able to compile a simple test program" happened on more than one of my machines, and I had installed previous versions by manually adjusting the Makefiles to use g++ instead of clang; but today I thought I'd try and fix things to work out-of-the-box...

maliasadi commented 1 year ago

Thanks for checking! I can feel the pain :cry: we're discussing the issue and should be able to come up with a better solution.

Such feedback helps us gain a deeper understanding of edge cases, and we're committed to working on robust solutions that make the process of installing Catalyst from source even smoother.

dime10 commented 1 year ago

Still investigating this, but we might need to recommend users not use conda when building from source.

dime10 commented 3 months ago

Hi @glassnotes, we actually have an update on this because the problem came up again. The issue is that conda ships with a version of the C++ standard library (libstdc++.so.6) that is older than what the Catalyst package was compiled with. In addition, when a conda environment is active it will put its own libraries ahead of the system libraries, so even though most modern distributions (say Ubuntu 20+) will have the right libraries conda injects an older one that is incompatible.

The solution is quite simple in this case, updating the relevant conda package with:

conda install -c conda-forge libstdcxx-ng

Hope this helps if you ever run into this issue again.

Just wanted to confirm, but it looks like a conda environment was active when you received the error:

ImportError: /home/olivia/.conda/envs/catalyst/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/olivia/Code/catalyst/frontend/catalyst/../../mlir/build/python_packages/quantum/mlir_quantum/compiler_driver.so)

is that correct?

Deactivating the environment would also solve the error from what I saw during testing.

PennyLaneAI / catalyst