OpenUSD fails to compile on Ubuntu22.04 due to missing Python symbols during linking

[stjaeckel]

Description of Issue

Build script fails when installing on Ubuntu 22.04. Setup:

• Ubuntu 22.04
• Python environment managed by anaconda
• All dependencies (Boost, OpenSubdiv, TBB, MaterialX) install without error

• Compiling OpenUSD fails with error:

  /usr/bin/ld: CMakeFiles/sdffilter.dir/sdffilter.cpp.o:sdffilter.cpp:(.text._ZN32pxrInternal_v0_24__pxrReserved__7VtValue13_TypeInfoImplISt3mapIdS0_St4lessIdESaISt4pairIKdS0_EEEN5boost13intrusive_ptrINS0_8_CountedIS9_EEEENS0_15_RemoteTypeInfoIS9_EEE9_GetPyObjERKNSt15aligned_storageILm8ELm8EE4typeE[_ZN32pxrInternal_v0_24__pxrReserved__7VtValue13_TypeInfoImplISt3mapIdS0_St4lessIdESaISt4pairIKdS0_EEEN5boost13intrusive_ptrINS0_8_CountedIS9_EEEENS0_15_RemoteTypeInfoIS9_EEE9_GetPyObjERKNSt15aligned_storageILm8ELm8EE4typeE]+0xb4): more undefined references to `_Py_Dealloc' follow
  /usr/bin/ld: CMakeFiles/sdffilter.dir/sdffilter.cpp.o: in function `_GLOBAL__sub_I_main':
  sdffilter.cpp:(.text.startup+0x294f): undefined reference to `_Py_NoneStruct'
  /usr/bin/ld: sdffilter.cpp:(.text.startup+0x295d): undefined reference to `_Py_NoneStruct'
  /usr/bin/ld: ../../../base/vt/libusd_vt.so: undefined reference to `PyObject_GetBuffer'
  /usr/bin/ld: ../../../base/tf/libusd_tf.so: undefined reference to `PyObject_Repr'
  /usr/bin/ld: ../../../base/tf/libusd_tf.so: undefined reference to `PyClassMethod_Type'
  /usr/bin/ld: ../../../base/vt/libusd_vt.so: undefined reference to `PyExc_ValueError'
  /usr/bin/ld: /home/stephan/RT/OpenUSD/install/lib/libboost_python310.so: undefined reference to `PyLong_AsLong'

  The issue seems to be a linking error. When checking the provided symbols on libboost_python310.so:

  
  nm -D /home/stephan/RT/OpenUSD/install/lib/libboost_python310.so | grep Py
  
               U PyArg_ParseTupleAndKeywords
               U PyBaseObject_Type
               U PyBool_FromLong
               U PyBool_Type
               U Py_BuildValue
               U PyBytes_AsString

...

The output from the nm command shows that libboost_python310.so references a large number of Python C API functions (those symbols prefixed with Py), but all of these symbols are marked with a U, indicating that they are undefined within the library itself. This means that libboost_python310.so expects these symbols to be provided by another binary at runtime—typically, this would be the Python interpreter (libpython) itself. However, when building OpenUSD, the python library is not linked. 

Manually building OpenUSD causes the same error (`undefined reference to `PyLong_AsLong'`):

cd ~ && rm -rf RT/MaterialX cd ~/RT && git clone --branch v1.38.9 https://github.com/AcademySoftwareFoundation/MaterialX cd ~/RT/MaterialX && mkdir build && mkdir install cmake -B "~/RT/MaterialX/build" -DCMAKE_INSTALL_PREFIX="~/RT/MaterialX/install" -DCMAKE_PREFIX_PATH="~/RT/MaterialX/install" -DCMAKE_BUILD_TYPE=Release -DMATERIALX_BUILD_SHARED_LIBS=ON -DMATERIALX_BUILD_TESTS=OFF cd build/ && make -j32 && make install

cd ~ && rm -rf RT/OpenSubdiv cd ~/RT && git clone --branch v3_6_0 https://github.com/PixarAnimationStudios/OpenSubdiv cd ~/RT/OpenSubdiv && mkdir build && mkdir install cmake -B "~/RT/OpenSubdiv/build" -DCMAKE_INSTALL_PREFIX="~/RT/OpenSubdiv/install" -DCMAKE_PREFIX_PATH="~/RT/OpenSubdiv/install" -DCMAKE_BUILD_TYPE=Release -DNO_EXAMPLES=ON -DNO_TUTORIALS=ON -DNO_REGRESSION=ON -DNO_DOC=ON -DNO_OMP=ON -DNO_CUDA=ON -DNO_OPENCL=ON -DNO_DX=ON -DNO_TESTS=ON -DNO_GLEW=ON -DNO_GLFW=ON -DNO_PTEX=ON -DNO_TBB=ON cd build/ && make -j32 && make install

cd ~ && rm -rf RT/OpenUSD cd ~/RT && git clone --branch v24.03 https://github.com/PixarAnimationStudios/OpenUSD cd ~/RT/OpenUSD && rm -rf build && rm -rf install && mkdir build && mkdir install

cmake -B "~/RT/OpenUSD/build" -DCMAKE_INSTALL_PREFIX="~/RT/OpenUSD/install" -DCMAKE_PREFIX_PATH="~/RT/OpenUSD/install" -DCMAKE_BUILD_TYPE=Release -DOPENSUBDIV_INCLUDE_DIR="/home/stephan/RT/OpenSubdiv/install/include" -DOPENSUBDIV_LIBRARIES="/home/stephan/RT/OpenSubdiv/install/lib" -DMaterialX_DIR="/home/stephan/RT/MaterialX/install/lib/cmake/MaterialX/"

cd build/ && make -j32 && make install

When checking the OpenUSD build config, there is a link to `libboost_python310.so`, but no link to `libpython3.10.so` which seems to be required for boost to work.

cmake -B "~/RT/OpenUSD/build" -LA

-- Cache values BUILD_SHARED_LIBS:BOOL=ON Boost_DIR:PATH=/home/stephan/anaconda3/envs/rt/lib/cmake/Boost-1.84.0 Boost_INCLUDE_DIR:PATH=/home/stephan/anaconda3/envs/rt/include Boost_LIBRARY_DIR_DEBUG:PATH=/home/stephan/anaconda3/envs/rt/lib Boost_LIBRARY_DIR_RELEASE:PATH=/home/stephan/anaconda3/envs/rt/lib Boost_NO_BOOST_CMAKE:BOOL=ON Boost_PYTHON310_LIBRARY_DEBUG:FILEPATH=/home/stephan/anaconda3/envs/rt/lib/libboost_python310.so Boost_PYTHON310_LIBRARY_RELEASE:FILEPATH=/home/stephan/anaconda3/envs/rt/lib/libboost_python310.so CHMODBIN:FILEPATH=/usr/bin/chmod CMAKE_ADDR2LINE:FILEPATH=/usr/bin/addr2line CMAKE_AR:FILEPATH=/usr/bin/ar CMAKE_BUILD_TYPE:STRING=Release CMAKE_COLOR_MAKEFILE:BOOL=ON CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++ ...

I tried different Python versions from 3.8 to 3.12 (all istalled using anaconda) and different OpenUSD versions from 23.01 to 24.03 from GitHUB. All attempts to compile OpenUSD fail due to the same issue. Using the default Ubuntu 22.04 system installer without anaconda fails too since the cmake version from the Ubuntu 22.04 repository is 3.22 and OpenUSD requires at least 3.24.

### Steps to Reproduce
1. Setup conda environment:

conda deactivate conda remove -n rt --all conda create --name rt -c conda-forge python=3.10 pyside6 pyopengl pybind11 cmake boost libpython-static conda activate rt

2. Download and try to compile OpenUSD

mkdir ~/RT cd ~/RT rm -rf OpenUSD/ git clone --branch v24.03 https://github.com/PixarAnimationStudios/OpenUSD cd ~/RT/OpenUSD/ && rm -rf install python3 build_scripts/build_usd.py install/ -v -v -v

### Package Versions

cmake 3.29.0 (from anaconda)
python 3.10.14 (from anaconda)
pyside6 6.6.2 (from anaconda)
boost 1.78 (installed from OpenUSD build script)

[optseb]

Hi @stjaeckel I'm new to OpenUSD building too, but my experience might help. I have built OpenUSD with the cmake process, which allows me to specify the python to use. I set up a virtual python environment (using the python3.11 that's installed on my Ubuntu 23.10 laptop), and then pip install pyside6 into this virtual environment. Then I call the OpenUSD build process with

cmake .. -DPYTHON_EXECUTABLE=/path/to/virtual/python/bin/python3.11 

Because I was using the package managed python, I was able to apt install pybind11-dev, which seemed to be required, possibly for one of the deps or to build boost with python support. I also had to make sure that when I built boost, I made sure to activate my virtual python first, and also make sure that the python api headers were in the CPLUS_INCLUDE_PATH environment variable.

source /path/to/virtual/python/bin/activate
export CPLUS_INCLUDE_PATH="$CPLUS_INCLUDE_PATH:/usr/include/python3.11"
# Now go ahead and build the boost library

Finally, although it may not affect you, Note that boost 1.78 has an incompatibility with Python 3.11+. This was fixed in boost 1.81.

[jesschimein]

Filed as internal issue #USD-9517