search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
29.02k stars 11.96k forks source link

LLDB build fails when python is not at the same prefix. #56675

Open Paul-Ferrell opened 2 years ago

Paul-Ferrell commented 2 years ago

The lldb/bindings/get-python-config.py script fails when the python lib path is not within lldb's prefix, and CMakeLists.txt for LLDB has no backup plan in that case.

The idea that the installed python shares a prefix with LLVM is not a reasonable assumption, particularly on systems where module systems (LMOD, TCL environment modules) are used to install multiple versions of LLVM in tandem. This also breaks installs with the Spack package manager, as all packages are installed in their own independent prefix.

Tagging @lawrence-danna-apple because he was responsible for the change that broke this.

Note: Where the heck do module systems and Spack get used? On supercomputers, mainly.

llvmbot commented 2 years ago

@llvm/issue-subscribers-lldb

labath commented 2 years ago

LLDB needs to have /some/ way of finding python. Could you give more detail about how the paths of the various components look like on the affected systems, and maybe how you would go about finding them (e.g. is hardcoding an absolute path a possibility) ?

Also note that it is possible to bypass the get-python-config.py script by defining the appropriate cmake variables manually. Could that maybe help in your case?

Paul-Ferrell commented 2 years ago

In the pre get-python-config.py way of handling things, LLDB got the python library paths and just kind of assumed they would work. Now it tries to force the creation of a path relative to the same install prefix as LLDB - that simply won't work, as the Python being built against could be anywhere on the system - it's literally unpredictable.

The good news is that the correct Python will always be the first one found via searching PATH (and LD_LIBRARY_PATH), and Cmake's find_package handles this all just fine. Similarly, the correct Python is found when using which or just running python3 on the terminal. So when python3 get-python-config.py is run by cmake the correct Python is being used and it will find the correct library paths - it's just that those paths won't necessarily have anything to do with where LLVM/LLDB is getting installed.

Side Note:

Spack install paths look like this: /usr/projects/hpcsoft/tce/spack/python/gcc-10.3.0/python-3.9.5-75wamw4vqe2zqx2nrzpmmtoxwcixbbtt/. The /usr/projects/hpcsoft/tce/spack part is the spack installation path, which could be anywhere. The giant hash in that path is derived from all the settings that went into building that package and all of its dependencies. This would be just one install of python amongst many in the same Spack environment. Similarly, when we go to install LLVM, it will have its own standalone prefix with a giant hash in its path, and it may be installed alongside other versions/configurations of LLVM.