Closed adro79 closed 1 year ago
I can't see any errors in your CMake log? Do note, though that gcc 6.3.1 is the highest gcc we test with.
That's weird.. Here's the terminal output:
-- Found Boost: /usr/include (found version "1.79.0")
-- Disabling boost-provided cmake config
-- Found Boost: /usr/include (found version "1.79.0") found components: python310
-- Found Jinja2
-- Found Boost: /usr/include (found version "1.79.0") found components: program_options
CMake Error at cmake/modules/FindTBB.cmake:200 (file):
file failed to open for reading (No such file or directory):
/usr/include/tbb/tbb_stddef.h
Call Stack (most recent call first):
cmake/defaults/Packages.cmake:135 (find_package)
CMakeLists.txt:23 (include)
CMake Deprecation Warning at cmake/defaults/Packages.cmake:207 (cmake_policy):
The OLD behavior for policy CMP0072 will be removed from a future version
of CMake.
The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
CMakeLists.txt:23 (include)
-- Found PySide6: with /usr/bin/python, will use /usr/bin/uic for pyside-uic binary
-- Found PyOpenGL
-- C++ namespace configured to (external) pxr, (internal) pxrInternal_v0_22
-- Skipping validation of gf generated code because PXR_VALIDATE_GENERATED_CODE=OFF
-- Skipping validation of sdf generated code because PXR_VALIDATE_GENERATED_CODE=OFF
-- Skipping alembic-based usddiff tests because PXR_BUILD_ALEMBIC_PLUGIN=OFF
-- Skipping Draco-based usddiff tests because PXR_BUILD_DRACO_PLUGIN=OFF
-- Skipping hgiMetal because PXR_BUILD_GPU_SUPPORT or PXR_ENABLE_METAL_SUPPORT is OFF
-- Skipping hgiVulkan because PXR_BUILD_GPU_SUPPORT or PXR_ENABLE_VULKAN_SUPPORT is OFF
-- Configuring incomplete, errors occurred!
See also "/home/adro/Repositories/USD/build/CMakeFiles/CMakeOutput.log".
Perhaps you need to install tbb, or tell cmake where to find it if you've got it somewhere other than where it knows to look?
CMake Error at cmake/modules/FindTBB.cmake:200 (file):
file failed to open for reading (No such file or directory):
/usr/include/tbb/tbb_stddef.h
Call Stack (most recent call first):
cmake/defaults/Packages.cmake:135 (find_package)
CMakeLists.txt:23 (include)
Our find script is here https://github.com/PixarAnimationStudios/USD/blob/release/cmake/modules/FindTBB.cmake and it lists up all the tbb related variables and how they should be set.
Okay, I did a little research of tbb_stddef.h and turns out the 2021.1.1.119 version of TBB changed the name of the file to version.h.
Replacing tbb_stddef.h to version.h in FindTBB.cmake solved the error.
This got me to the next step, that ended with some errors as well:
In file included from /home/adro/Repositories/USD/pxr/base/trace/collector.h:31,
from /home/adro/Repositories/USD/pxr/base/trace/collector.cpp:25:
/home/adro/Repositories/USD/pxr/base/trace/concurrentList.h: In instantiation of ‘pxrInternal_v0_22__pxrReserved__::TraceConcurrentList<T>::iterator pxrInternal_v0_22__pxrReserved__::TraceConcurrentList<T>::Insert() [with T = pxrInternal_v0_22__pxrReserved__::TraceCollector::_PerThreadData]’:
/home/adro/Repositories/USD/pxr/base/trace/collector.cpp:67:49: required from here
/home/adro/Repositories/USD/pxr/base/trace/concurrentList.h:133:16: error: ‘class tbb::detail::d1::cache_aligned_allocator<pxrInternal_v0_22__pxrReserved__::TraceConcurrentList<pxrInternal_v0_22__pxrReserved__::TraceCollector::_PerThreadData>::Node>’ has no member named ‘construct’
133 | _alloc.construct(newNode);
| ~~~~~~~^~~~~~~~~
/home/adro/Repositories/USD/pxr/base/trace/concurrentList.h: In instantiation of ‘pxrInternal_v0_22__pxrReserved__::TraceConcurrentList<T>::~TraceConcurrentList() [with T = pxrInternal_v0_22__pxrReserved__::TraceCollector::_PerThreadData]’:
/home/adro/Repositories/USD/pxr/base/trace/collector.cpp:82:36: required from here
/home/adro/Repositories/USD/pxr/base/trace/concurrentList.h:114:20: error: ‘class tbb::detail::d1::cache_aligned_allocator<pxrInternal_v0_22__pxrReserved__::TraceConcurrentList<pxrInternal_v0_22__pxrReserved__::TraceCollector::_PerThreadData>::Node>’ has no member named ‘destroy’
114 | _alloc.destroy(nodeToDelete);
| ~~~~~~~^~~~~~~
make[2]: *** [pxr/base/trace/CMakeFiles/trace.dir/build.make:357: pxr/base/trace/CMakeFiles/trace.dir/collector.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:3722: pxr/base/trace/CMakeFiles/trace.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
USD is built against tbb 2019 update 6 on Linux. Unfortunately, the API for tbb has changed rapidly in recent years and removed functionality that USD relies on. USD is built against the vfx reference platform (https://vfxplatform.com) which as you can see has got old distributions of tbb in the pipeline for some time. This is due to a need to be compatible with applications such as Maya and Houdini which expect olden tbbs.
I'm sorry that I don't have a better answer than this: You could build tbb 2019 U6 into a new location, and coerce findtbb into finding it.
Other options, such as modifying the USD source code to be compatible with newer tbb's are possible, but most likely are a lot of work. That work needs to happen eventually, but as you can see from the vfxplatform schedule, it isn't likely to occur in a time frame that helps you today.
I'm also trying to package this but, even with legacy TBB2019.6 (I've created an AUR tbb2019), is not building.
Linking CXX executable pxr/usd/bin/sdfdump/sdfdump
FAILED: pxr/usd/bin/sdfdump/sdfdump
: && /usr/bin/c++ -Wall -Wformat-security -pthread -Wno-deprecated -Wno-deprecated-declarations -Wno-unused-local-typedefs -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -DBOOST_BIND_GLOBAL_PLACEHOLDERS -O3 -DNDEBUG -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now pxr/usd/bin/sdfdump/CMakeFiles/sdfdump.dir/sdfdump.cpp.o -o pxr/usd/bin/sdfdump/sdfdump -Wl,-rpath,/home/maciel/Downloads/Software/usd/src/build: libusd_ms.so /usr/lib/libboost_program_options.so /usr/lib/libjemalloc.so /usr/lib/libjemalloc.so -ldl -lm /usr/lib/libpython3.10.so /usr/lib/libboost_python310.so -ltbb /usr/lib/libSM.so /usr/lib/libICE.so /usr/lib/libX11.so /usr/lib/libXext.so /usr/lib/libGL.so /usr/lib/libosdCPU.so /usr/lib/libosdGPU.so /lib64/libPtex.so /usr/lib/libpython3.10.so /usr/lib/libboost_python310.so -ltbb /usr/lib/libSM.so /usr/lib/libICE.so /usr/lib/libX11.so /usr/lib/libXext.so /usr/lib/libGL.so /usr/lib/libosdCPU.so /usr/lib/libosdGPU.so /lib64/libPtex.so /usr/lib/libjemalloc.so && :
/usr/bin/ld: warning: libtbb.so.12, needed by /usr/lib/libosdCPU.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::create(tbb::detail::d1::global_control&)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::allocate(tbb::detail::d1::small_object_pool*&, unsigned long, tbb::detail::d1::execution_data const&)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::notify_waiters(unsigned long)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::initialize(tbb::detail::d1::task_group_context&)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::is_group_execution_cancelled(tbb::detail::d1::task_group_context&)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::assertion_failure(char const*, int, char const*, char const*)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::deallocate(tbb::detail::d1::small_object_pool&, void*, unsigned long, tbb::detail::d1::execution_data const&)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::execution_slot(tbb::detail::d1::execution_data const*)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::destroy(tbb::detail::d1::task_group_context&)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::destroy(tbb::detail::d1::global_control&)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::max_concurrency(tbb::detail::d1::task_arena_base const*)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::spawn(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&)'
/usr/bin/ld: /usr/lib/libosdCPU.so: undefined reference to `tbb::detail::r1::allocate(tbb::detail::d1::small_object_pool*&, unsigned long)'
collect2: error: ld returned 1 exit status
looking carefully to warning: libtbb.so.12, needed by /usr/lib/libosdCPU.so
, it seams that we will need downgrade opensubdiv
(?!) as well, but I'm not sure, I'd assume that usd-22.08
is in pair with the latest opensubdiv-3.4.4
Tbb 2019.6 provides libtbb.so.2
; way behind 12
.
I finally built USD successfully! As @meshula said, TBB was the only problem that I had for a minimal USD Build.
This is my final cmake config:
cmake .. \
-DCMAKE_INSTALL_PREFIX:PATH=USD/INSTALL/PATH \
-DTBB_ROOT_DIR=TBB/2019.6/PATH \
-DBoost_NO_BOOST_CMAKE=ON \
-DBUILD_SHARED_LIBS=ON \
-DPXR_USE_PYTHON_3=ON \
-DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python \
-DPYTHON_INCLUDE_DIR=/usr/include/python3.10 \
-DPXR_MALLOC_LIBRARY:path=/usr/lib/libjemalloc.so \
However, when trying to build all plugins I encountered that OpenImageIO and Alembic plugins failed to build. Here's the cmake Output of one of the errors:
CMake Warning at cmake/macros/Private.cmake:885 (target_link_libraries):
Target "usdAbc" requests linking to directory
"/home/adro/Repositories/USD/build/openexr-2.4.3/build/IlmBase/Half".
Targets may link only to libraries. CMake is dropping the item.
Call Stack (most recent call first):
cmake/macros/Private.cmake:1396 (_pxr_target_link_libraries)
cmake/macros/Public.cmake:327 (_pxr_library)
cmake/macros/Public.cmake:367 (pxr_library)
pxr/usd/plugin/usdAbc/CMakeLists.txt:18 (pxr_plugin)
I'll try later using the supported versions.
Thanks for the help!!
@adro79 , can you build with -DPXR_BUILD_MONOLITHIC=ON
?
I think it will be needed to integrate with blender afterwards.
Yes, I'm able to build it.
Unlike you, i used the prebuilt binaries of TBB, maybe something's missing in your TBB build config.
I'll keep digging into it. The prebuilt binaries were compiled with gcc4.7
.
It's just too ancient to package a monolithic build, given that we''re building with gcc12.1
.
USD is built against OpenSubdiv 3.4.4.
I think I figured what is going on. Arch's opensubdiv
was patched to work with tbb-2021
.
So, when I try to build usd
with tbb2019
and -DPXR_BUILD_MONOLITHIC=ON
, the linker goes crazy.
The issue goes away building opensubdiv
also with tbb2019
.
So, since we need monolithic to integrate with blender, I think creating an AUR variant opensubdiv-tbb2019
will be the solution.
Or, perhaps, build both tbb2019
and opensubdiv-tbb2019
on prepare()
before build()
in usd
' s PKGBUILD.
Yeah, I think building those package in prepare is the cleanest option. Furthermore, they're only required for building USD afaik.
I'm also thinking how to give the option of enabling plugin support to the build. For example, you only need a Monolithic build for building Blender, but I'd like to have the option of having the Embree Renderer and MaterialX support.
I'm not that experienced with AUR and building from source, but I can help with anything.
Yeah, I think building those package in prepare is the cleanest option. Furthermore, they're only required for building USD afaik.
I've package it in a single PKGBUILD now :
I'm also thinking how to give the option of enabling plugin support to the build. For example, you only need a Monolithic build for building Blender, but I'd like to have the option of having the Embree Renderer and MaterialX support.
It would be a nice addition! Now that we can package it, we're in a better position to bring more eyes (e.g., from blender or embree people) to this.
I was trying to build blender with usd, but I stumbled upon tbb issues. :( I'm starting to think that we will need to make it work with tbb2021 to integrate with Arch.
On top of that, as you can see in the PKGBUILD above, I'm also running the automated tests. But, look at my build log:
3% tests passed, 904 tests failed out of 930
I'm running out of ideas here.
Could you run one of the tests manually and check the log? Usually when every single test fails, there's often one single issue, and resolving that gets you going.
The two most common "all failed" situations are ~
the pluginfo.json file wasn't found, most likely the Plug library isn't looking where you installed the json files. The quickest way to understand that point is to have a look at this code, and if a resolution isn't obvious, try breakpointing _AppendPathList and visually inspect the paths it's checking
Python got initialized twice, and every test is failing with the good old PyInitialize assert. This is a common symptom of running with a statically linked Python. I'm not sure how Blender builds Python. See this recent discussion ~ https://github.com/PixarAnimationStudios/USD/issues/1996
Could you run one of the tests manually and check the log? Usually when every single test fails, there's often one single issue, and resolving that gets you going.
Indeed, @meshula , it's not setting --testenv-dir
correctly. It's pointing to /usr
, where it should have pointed to my build dir.
Do I need to set -DCMAKE_INSTALL_PREFIX=
before building? I usually set DESTDIR
afterwards to package in an appropriate folder.
I'm getting something similar to this in all failed tests:
----------------------------------------------------------
1/930 Testing: testArchAbi
1/930 Test: testArchAbi
Command: "/usr/bin/python" "/home/maciel/Downloads/Software/usd/src/USD/cmake/macros/testWrapper.py" "--verbose" "--testenv-dir=/usr/tests/ctest/testArchAbi" "--env-var=PYTHONPATH=/usr/lib/python:" "/usr/tests/testArchAbi"
Directory: /home/maciel/Downloads/Software/usd/src/build/pxr/base/arch
"testArchAbi" start time: Aug 22 14:33 -03
Output:
----------------------------------------------------------
chdir: /tmp/tmpww02x7rz
cmd: ['/usr/tests/testArchAbi']
Traceback (most recent call last):
File "/home/maciel/Downloads/Software/usd/src/USD/cmake/macros/testWrapper.py", line 387, in <module>
_runCommand(args.cmd, args.stdout_redirect, args.stderr_redirect,
File "/home/maciel/Downloads/Software/usd/src/USD/cmake/macros/testWrapper.py", line 306, in _runCommand
retcode = _convertRetCode(subprocess.call(cmd, shell=False, env=env,
File "/usr/lib/python3.10/subprocess.py", line 345, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.10/subprocess.py", line 969, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.10/subprocess.py", line 1845, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/tests/testArchAbi'
<end of output>
Test time = 0.03 sec
----------------------------------------------------------
Test Failed.
"testArchAbi" end time: Aug 22 14:33 -03
"testArchAbi" time elapsed: 00:00:00
----------------------------------------------------------
The two most common "all failed" situations are ~
* the pluginfo.json file wasn't found, most likely the Plug library isn't looking where you installed the json files. The quickest way to understand that point is to have a look at this code, and if a resolution isn't obvious, try breakpointing _AppendPathList and visually inspect the paths it's checking * Python got initialized twice, and every test is failing with the good old PyInitialize assert. This is a common symptom of running with a statically linked Python. I'm not sure how Blender builds Python. See this recent discussion ~ [USD release 22.08 compiles, but any CLI invocation crashes on an M1 Mac #1996](https://github.com/PixarAnimationStudios/USD/issues/1996)
I see, I'll consider this after solving the previous issue. Thank you for pointing this out.
@qumaciel Yes, the build system sets the testenv-dir based on CMAKE_INSTALL_PREFIX so you'll need to set that at cmake time.
After fixing the prefix and adding LD_PRELOAD=/usr/lib/libjemalloc.so ninja -C build test
, I've managed to clear almost everything!
Btw, I thought I could run everything with PySide6. The only issue I've got was, e.g.,
885/930 Testing: testUsdviewTimeSamples
885/930 Test: testUsdviewTimeSamples
Command: "/usr/bin/python" "/home/maciel/Downloads/Software/usd/src/USD/cmake/macros/testWrapper.py" "--verbose" "--testenv-dir=/home/maciel/Downloads/Software/usd/pkg/usd/tests/ctest/testUsdviewTimeSamples" "--env-var=PYTHONPATH=/home/maciel/Downloads/Software/usd/pkg/usd/lib/python:" "/usr/bin/python /home/maciel/Downloads/Software/usd/pkg/usd/bin/testusdview --testScript testUsdviewTimeSamples.py test.usda"
Directory: /home/maciel/Downloads/Software/usd/src/build/pxr/usdImaging/bin/testusdview
"testUsdviewTimeSamples" start time: Aug 22 20:13 -03
Output:
----------------------------------------------------------
Traceback (most recent call last):
File "/home/maciel/Downloads/Software/usd/pkg/usd/bin/testusdview", line 33, in <module>
import pxr.Usdviewq as Usdviewq
File "/home/maciel/Downloads/Software/usd/pkg/usd/lib/python/pxr/Usdviewq/__init__.py", line 32, in <module>
from .qt import QtWidgets, QtCore
File "/home/maciel/Downloads/Software/usd/pkg/usd/lib/python/pxr/Usdviewq/qt.py", line 42, in <module>
PySideModule = GetPySideModule()
File "/home/maciel/Downloads/Software/usd/pkg/usd/lib/python/pxr/Usdviewq/qt.py", line 31, in GetPySideModule
from . import attributeValueEditorUI
File "/home/maciel/Downloads/Software/usd/pkg/usd/lib/python/pxr/Usdviewq/attributeValueEditorUI.py", line 11, in <module>
from PySide2.QtCore import * # type: ignore
ModuleNotFoundError: No module named 'PySide2'
Error: return code 1 doesn't match expected 0 (EXPECTED_RETURN_CODE).chdir: /tmp/tmplhqb0k49
copying testenv dir: /home/maciel/Downloads/Software/usd/pkg/usd/tests/ctest/testUsdviewTimeSamples
cmd: ['/usr/bin/python', '/home/maciel/Downloads/Software/usd/pkg/usd/bin/testusdview', '--testScript', 'testUsdviewTimeSamples.py', 'test.usda']
<end of output>
Test time = 0.18 sec
----------------------------------------------------------
Test Failed.
"testUsdviewTimeSamples" end time: Aug 22 20:13 -03
"testUsdviewTimeSamples" time elapsed: 00:00:00
----------------------------------------------------------
It also occurs with other usdview tests.
Anyway, now I can start tweaking with tbb. Thank you!
ModuleNotFoundError: No module named 'PySide2'
Seems like you're missing PySide2, try
pip install pyside2
and check if there's a warning pointing out that you need to add it in PATH.
Fantastic progress!
We did recently land support for Pyside6, but it's possible that we neglected the unit tests, that needs to be corrected if we did miss it. Pyside6 works with usdview, but as suggested, running with Pyside2 for now will get you past that.
Filed as internal issue #USD-7577
@qumaciel did you manage to compile USD with your PKGBUILD? If not, I have some spare time to take a look at it if you want. (I'm not an expert but it's better than nothing)
USD is built against tbb 2019 update 6 on Linux. Unfortunately, the API for tbb has changed rapidly in recent years and removed functionality that USD relies on. USD is built against the vfx reference platform (https://vfxplatform.com) which as you can see has got old distributions of tbb in the pipeline for some time. This is due to a need to be compatible with applications such as Maya and Houdini which expect olden tbbs.
I'm sorry that I don't have a better answer than this: You could build tbb 2019 U6 into a new location, and coerce findtbb into finding it.
Other options, such as modifying the USD source code to be compatible with newer tbb's are possible, but most likely are a lot of work. That work needs to happen eventually, but as you can see from the vfxplatform schedule, it isn't likely to occur in a time frame that helps you today.
Hi, I'm looking to build USD for AArch64 on Windows, but it looks like tbb 2019 U6 doesn't have AArch64 support. oneTBB latest does, so what does the time frame for oneTBB support look like?
Hi Stephen, We can’t yet say when we’ll be supporting OneAPI, and it will likely be awhile. But we will be moving to CY 2022 of the vfx reference platform, hopefully in the first half of next year. That brings TBB 2020.3, which does look like it supports an AArch64 build.
On Tue, Oct 18, 2022 at 10:36 AM Stephen Long @.***> wrote:
USD is built against tbb 2019 update 6 on Linux. Unfortunately, the API for tbb has changed rapidly in recent years and removed functionality that USD relies on. USD is built against the vfx reference platform ( https://vfxplatform.com) which as you can see has got old distributions of tbb in the pipeline for some time. This is due to a need to be compatible with applications such as Maya and Houdini which expect olden tbbs.
I'm sorry that I don't have a better answer than this: You could build tbb 2019 U6 into a new location, and coerce findtbb into finding it.
Other options, such as modifying the USD source code to be compatible with newer tbb's are possible, but most likely are a lot of work. That work needs to happen eventually, but as you can see from the vfxplatform schedule, it isn't likely to occur in a time frame that helps you today.
Hi, I'm looking to build USD for AArch64 on Windows, but it looks like tbb 2019 U6 doesn't have AArch64 support. oneTBB latest does, so what does the time frame for oneTBB support look like?
— Reply to this email directly, view it on GitHub https://github.com/PixarAnimationStudios/USD/issues/2000#issuecomment-1282768513, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPOU2CJ2GN4UKQ5LGLBFPLWD3N2PANCNFSM5666E6OA . You are receiving this because you commented.Message ID: @.***>
-- --spiffiPhone
@adro79 : I had to install tbb2019 from the aur to resolve the construct
destroy
issue.
I'm trying to build USD using gcc 12.1.1, cmake 3.24 and python 3.10.6
System Information (OS, Hardware) Linux 5.19.2-zen1-1-zen AMD Ryzen 7 5800X
My build flags are:
Here's the full CMake Log
Thanks in advance.