Conflict with Houdini USD and MoonRay USD?

[BrianHanke]

I'm splitting this off from our previous discussion since I've narrowed down the problem a lot.

System info: CentOS 7, MoonRay compiled using default settings from build guide, using Python 3.9.10 to match Houdini, Houdini FX 19.5.534.

See attached video. First is a quick test of moonray_gui to confirm the renderer is working. Then, launching usdview from hython, Karma works but as you can see MoonRay throws a ton of errors about "multiple debug symbol" and "already defined C++ type."

It looks to me (as a non-coder) like the system is trying to load both MoonRay and Houdini USD libraries? Any ideas how to resolve this?

https://user-images.githubusercontent.com/59420805/227555316-9f4765af-9c62-4e62-92fc-6c991919860a.mp4

[BrianHanke]

Made some progress by symlinking all the "libusd" libraries to the "libpxr" ones that come with Houdini. That eliminated 99% of the errors. Still not working though. Now we're left with:

Error in 'pxrInternal_v0_22pxrReserved::PlugPlugin::_Load' at line 257 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/plug/plugin.cpp : 'Failed to load plugin 'hdMoonray': /installs/openmoonray/plugin/hdMoonray.so: undefined symbol: _ZN32pxrInternal_v0_22pxrReserved14TfPyObjWrapperC1EN5boost6python3api6objectE in '/installs/openmoonray/plugin/hdMoonray.so''
Error in 'pxrInternal_v0_22pxrReserved::HdRendererPluginRegistry::CreateRenderDelegate' at line 100 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rendererPluginRegistry.cpp : 'Couldn't find plugin for id HdMoonrayRendererPlugin'

Honestly this is all kind of bananas. Is there anybody at DreamWorks who could lend a helping hand? I don't think this should be this complicated and I feel like we're all flying blind out here. Thanks!

[vvzen]

@BrianHanke the

hdMoonray.so: undefined symbol: _ZN32pxrInternal_v0_22pxrReserved14TfPyObjWrapperC1EN5boost6python3api6objectE in '/installs/openmoonray/plugin/hdMoonray.so

makes me think of an API mismatch between libraries.

My first suggestion would be to look at which version of USD Houdini is using. For example, 19.5 is on 22.05: https://www.sidefx.com/docs/houdini/licenses/index.html . Then, check if moonray is using the same version. According to https://github.com/dreamworksanimation/openmoonray/blob/9b629e0999bfe2394ad56d31c3314c5199c4db3c/building/CMakeLists.txt#L205, Moonray is also targeting 22.05.

In that case, I think that it would be good to have a dedicated guide on how to build Moonray as a DCC plugin, since in that scenario it would be safer/easier to rely on the USD and OCIO offered by the DCC, instead of having the build script compile and link to a different USD. As long as openmoonray and the DCC are on the same VFX reference platform year, most of the versions of these big libraries should be in line (even though sadly that beast of USD isn't yet on the VFX ref platform, afaik..)

[BrianHanke]

API mismatch between libraries.

Yes indeed, that's my conclusion too. I tried one more test, symlinking all the Houdini libraries before the final MR build step. No library errors when doing that, but I started getting syntax/API problems.

SideFX recommended that I build against the Houdini USD libraries and the professional coders I've talked to agree. It's obviously a big task and above my skill level. Not much to do until DreamWorks throws us a bone...

[rgwilson68]

There are a couple of possible solutions. You should be able to get USD from the sidefx repo (https://github.com/sideeffects/USD) instead of pxr by changing the location in the dependency CMake project. You'll have to change the git tag, but you can use symbolic tags like 'v22.05a' -- there also seem to be tags for some specific Houdini versions on that repo.

Another way, that is more involved, is to use the USD libs and headers shipped with Houdini. To do this, you need a pxrConfig.cmake that targets these. I've uploaded a tar file with a modified set of config files and a README to: https://docs.openmoonray.org/assets/build_extras/moonray_houdini_pxr_config.tar -- I think you should be able to access this. You'll have to modify the CMake preset used to build MoonRay itself to get this working.

[BrianHanke]

@rgwilson68 Excellent, thanks! Will check this out over the next few days.

[BrianHanke]

Made a new build with the changes in option number 2 above. Build went fine, but still no rendering. Houdini crashes after clicking on MoonRay with the following log. usdview throws even more errors, a sample of which I put further down. It's just the same few ones over and over, but hundreds if not thousands of lines.

Crash report from brian; Houdini FX Version 19.5.534 [linux-x86_64-gcc9.3] Uptime 18 seconds Mon Mar 27 17:42:54 2023 Caught signal 11

Traceback from 29463 ThreadId=0x7f683ca39700 AP_Interface::si_CrashHandler::handledSignal(UTsignalHandlerArg) signalCallback(UTsignalHandlerArg) UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t, void) const UT_Signal::processSignal(int, siginfo_t, void) _L_unlock_13 hdMoonray::Light::Sync(pxrInternal_v0_22pxrReserved::HdSceneDelegate, pxrInternal_v0_22pxrReserved::HdRenderParam, unsigned int) pxrInternal_v0_22pxrReserved::Hd_PrimTypeIndex::_RenderDelegateDestroyPrim(pxrInternal_v0_22pxrReserved::HdRenderDelegate, pxrInternal_v0_22pxrReserved::HdBprim) pxrInternal_v0_22pxrReserved::HdRenderIndex::SyncAll(std::vector<std::shared_ptr, std::allocator<std::shared_ptr > >, std::unordered_map<pxrInternal_v0_22pxrReserved::TfToken, pxrInternal_v0_22pxrReserved::VtValue, pxrInternal_v0_22pxrReserved::TfToken::HashFunctor, std::equal_to, std::allocator<std::pair<pxrInternal_v0_22pxrReserved::TfToken const, pxrInternal_v0_22pxrReserved::VtValue> > >) pxrInternal_v0_22pxrReserved::XUSD_ImagingEngineGL::DispatchRender(UT_StringHolder const&, pxrInternal_v0_22pxrReserved::UsdPrim const&, pxrInternal_v0_22pxrReserved::XUSD_ImagingRenderParams const&) HUSD_Imaging::updateRenderData(UT_Matrix4T const&, UT_Matrix4T const&, UT_Rect const&, bool) tbb::internal::function_task<HUSD_Imaging::launchBackgroundRender(UT_Matrix4T const&, UT_Matrix4T const&, UT_Rect const&, UT_StringRef const&, UT_Options const, bool)::{lambda()#1}>::execute() tbb::internal::custom_scheduler::process_bypass_loop(tbb::internal::context_guard_helper&, tbb::task, long) (custom_scheduler.h:474) tbb::internal::custom_scheduler::local_wait_for_all(tbb::task&, tbb::task) (custom_scheduler.h:636) tbb::internal::arena::process(tbb::internal::generic_scheduler&) (arena.cpp:196) tbb::internal::market::process(rml::job&) (market.cpp:667) tbb::internal::rml::private_worker::run() (private_server.cpp:266) tbb::internal::rml::private_worker::thread_routine(void*) (private_server.cpp:219) start_thread __clone

Traceback from 29463 ThreadId=0x7f68674fff40 AP_Interface::coreDumpChaser(UTsignalHandlerArg) AP_Interface::si_CrashHandler::chaser(UTsignalHandlerArg) signalCallback(UTsignalHandlerArg) UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t, void) const UT_Signal::processSignal(int, siginfo_t, void) _L_unlock_13 GI_ppoll qt_safe_poll(pollfd, unsigned long, timespec const) QEventDispatcherUNIX::processEvents(QFlags) QXcbNativeInterface::dumpNativeWindows(unsigned long long) const UIQtGeneratorThread::processEvents(QFlags) UI_QtGeneratorThread::waitForRawEvents() UI_Queue::waitForGenerators(int) UI_Queue::waitForEvents() UI_Queue::generateEvents(bool, bool) UI_Queue::eventLoop() main libc_start_main main

---

Error in 'pxrInternal_v0_22pxrReserved::TfType::_DefineCppType' at line 963 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/tf/type.cpp : 'TfType 'SdfVariantSpec' already has a defined C++ type; cannot redefine' Error in 'pxrInternal_v0_22pxrReserved::Vt_CastRegistry::Register' at line 137 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/vt/value.cpp : 'VtValue cast already registered from 'string' to 'SdfAssetPath'. New cast will be ignored.' Error in 'pxrInternal_v0_22pxrReserved::Vt_CastRegistry::Register' at line 137 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/vt/value.cpp : 'VtValue cast already registered from 'double' to 'SdfTimeCode'. New cast will be ignored.' Error in 'pxrInternal_v0_22pxrReserved::Vt_CastRegistry::Register' at line 137 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/vt/value.cpp : 'VtValue cast already registered from 'SdfTimeCode' to 'double'. New cast will be ignored.' USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::HdRprim::_ReprDescConfigs<DESC_TYPE, N>::Find [with DESC_TYPE = pxrInternal_v0_22pxrReserved::HdMeshReprDesc; int, N = 2]' at line 329 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/imaging/hd/rprim.h : 'Repr smoothHull not found' Error in 'pxrInternal_v0_22pxrReserved::Tf_PyOwnershipRefBaseUniqueChanged' at line 402 in file /home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/usd/usd-22.05/USD-py3.9/pxr/base/tf/pyIdentity.cpp : 'Couldn't get uniqueId associated with refBase!'

[rgwilson68]

Is it possible that the pxr USD build is still on the system, and being picked up by cmake? You might need to check under /installs and /usr/local for pxrConfig.cmake, and remove it...

[BrianHanke]

Ah ok, I deleted all that and all the libusd* files, but I didn't see pxrConfig.cmake. Trying again!

[BrianHanke]

Do I need to delete PXR_USD_LOCATION and PXR_INCLUDE_DIRS under both container-environment and dwa-environment?

[rgwilson68]

No, dwa-environment is only used by internal dwa builds, so you don't need to change it.

[BrianHanke]

IT WORKS!!!

[BrianHanke]

Thanks so much for your amazing help Rob, much appreciated!

[nospitters]

@BrianHanke congratulations... would it be possible to share how you got it work? maybe build instructions?

[BrianHanke]

@nospitters Definitely. I'm working on a guide right now, will probably be ready this weekend. Check out my GitHub to stay updated.

[solomon-s2020]

Crash report from yu; Houdini FX Version 19.5.534 [linux-x86_64-gcc9.3] Uptime 135 seconds Sun Apr 9 16:55:40 2023 Caught signal 11

Traceback from 6705 ThreadId=0x7f4e87fff700 AP_Interface::si_CrashHandler::handledSignal(UTsignalHandlerArg) signalCallback(UTsignalHandlerArg) UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t, void) const UT_Signal::processSignal(int, siginfo_t, void) _L_unlock_13 pxrInternal_v0_22pxrReserved::HdChangeTracker::MarkSprimDirty(pxrInternal_v0_22pxrReserved::SdfPath const&, unsigned int) hdMoonray::RenderDelegate::setDisableLighting(bool) hdMoonray::RenderSettings::apply() hdMoonray::RenderDelegate::getRendererApplySettings() hdMoonray::RenderBuffer::Allocate(pxrInternal_v0_22pxrReserved::GfVec3i const&, pxrInternal_v0_22pxrReserved::HdFormat, bool) pxrInternal_v0_22pxrReserved::HdRenderBuffer::Sync(pxrInternal_v0_22pxrReserved::HdSceneDelegate, pxrInternal_v0_22pxrReserved::HdRenderParam, unsigned int) pxrInternal_v0_22pxrReserved::Hd_PrimTypeIndex::_RenderDelegateDestroyPrim(pxrInternal_v0_22pxrReserved::HdRenderDelegate, pxrInternal_v0_22pxrReserved::HdBprim) pxrInternal_v0_22pxrReserved::HdRenderIndex::SyncAll(std::vector<std::shared_ptr, std::allocator<std::shared_ptr > >, std::unordered_map<pxrInternal_v0_22pxrReserved::TfToken, pxrInternal_v0_22pxrReserved::VtValue, pxrInternal_v0_22pxrReserved::TfToken::HashFunctor, std::equal_to, std::allocator<std::pair<pxrInternal_v0_22pxrReserved::TfToken const, pxrInternal_v0_22pxrReserved::VtValue> > >) pxrInternal_v0_22pxrReserved::XUSD_ImagingEngineGL::DispatchRender(UT_StringHolder const&, pxrInternal_v0_22pxrReserved::UsdPrim const&, pxrInternal_v0_22pxrReserved::XUSD_ImagingRenderParams const&) HUSD_Imaging::updateRenderData(UT_Matrix4T const&, UT_Matrix4T const&, UT_Rect const&, bool) tbb::internal::function_task<HUSD_Imaging::launchBackgroundRender(UT_Matrix4T const&, UT_Matrix4T const&, UT_Rect const&, UT_StringRef const&, UT_Options const, bool)::{lambda()#1}>::execute() tbb::internal::custom_scheduler::process_bypass_loop(tbb::internal::context_guard_helper&, tbb::task, long) (custom_scheduler.h:474) tbb::internal::custom_scheduler::local_wait_for_all(tbb::task&, tbb::task) (custom_scheduler.h:636) tbb::internal::arena::process(tbb::internal::generic_scheduler&) (arena.cpp:196) tbb::internal::market::process(rml::job&) (market.cpp:667) tbb::internal::rml::private_worker::run() (private_server.cpp:266) tbb::internal::rml::private_worker::thread_routine(void*) (private_server.cpp:219) start_thread __clone

Traceback from 6705 ThreadId=0x7f4eb51aaf40 AP_Interface::coreDumpChaser(UTsignalHandlerArg) AP_Interface::si_CrashHandler::chaser(UTsignalHandlerArg) signalCallback(UTsignalHandlerArg) UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t, void) const UT_Signal::processSignal(int, siginfo_t, void) _L_unlock_13 GI_ppoll qt_safe_poll(pollfd, unsigned long, timespec const) QEventDispatcherUNIX::processEvents(QFlags) QXcbNativeInterface::dumpNativeWindows(unsigned long long) const UIQtGeneratorThread::processEvents(QFlags) UI_QtGeneratorThread::waitForRawEvents() UI_Queue::waitForGenerators(int) UI_Queue::waitForEvents() UI_Queue::generateEvents(bool, bool) UI_Queue::eventLoop() main libc_start_main main

Error Log.
Unable to successfully run rendering。