Double-loading USD on MacOS

[ikolev21]

Description of Issue

The USD library cannot be loaded twice into the same process on the MacOS (and probably Linux, not tested yet). It does work on Windows.

This is a continuation of #1341, which was the first problem we ran into. We found an acceptable workaround for the TBB deadlock.

The new problem is that when the second copy of USD gets loaded into the process, it exits the process with FATAL ERROR: [TF_DEBUG_ENVIRONMENT_SYMBOL] multiple symbol definitions from line 96 in pxr/base/tf/debug.cpp

What happens is approximately this:

• Cinema 4D R23 loads its own USD module, a shared library that links dynamically to libusd_ms.dylib, which does its registration.
• C4D loads our plugin (shared library), which loads another of our shared libraries that is statically linked to USD
• The copy of USD in our shared library registers its classes, but then it also triggers the registration in C4D's libusd_ms.dylib a second time, which causes the TF_FATAL_ERROR in debug.cpp.

This seems to be related to the DYLD load callback that gets registered by InstallDyldCallbacks() in pxr\base\arch\attributes.cpp. The first copy's callback gets called for the second copy that gets loaded into the process.

Any suggestions how to work around this?

One solution we can see is to link dynamically to USD and use C4D's copy of the library instead of carrying our own. But we don't want to be tied to the USD version carried by C4D, so this will only work if USD provides full backward compatibility. Then we could apply to it the approach we apply to TBB (see my comment in issue #1341): use an old version that's lower than the USD versions on all hosts that carry USD (and all 3 hosts we publish our plugin for are currently bringing in USD support). However, it's very likely that the hosts won't all use the same kind of linking to USD (like monolithic vs. non-monolithic) and we'll have to build a specific copy of our shared library for each host, which will bring another complication to our pretty complex build scripts.

We would prefer to find a solution with static linking to USD, isolating us from the other possible versions loaded into the process.

I wonder how the Windows implementation works without a DLL loading callback, can't the same approach work on Unix too?

Steps to Reproduce

1. Basically, try to load two shared objects each linked statically to USD into the same process.

System Information (OS, Hardware)

MacOS, and probably Linux (not tested). Doesn't happen on Windows.

Package Versions

20.02, 21.02

Build Flags

Static monolithic

[jtran56]

Filed as internal issue #USD-6616

[ikolev21]

I found an workaround: rebuilt USD after renaming the section tags pxrctor/pxrdtor (which btw would maybe be better defined as macro constants). But it's an ugly workaround, as any other that needs to modify (and even not trivially) the library distribution. (btw we have a few other such modifications, I'll post them in a separate issue)

[c64kernal]

Hi @ikolev21 -- we're not 100% sure this would solve your problem, but our intention was that using the namespacing options (see https://github.com/PixarAnimationStudios/USD/blob/release/BUILDING.md#usd-developer-options) would be for situations like these. Did that not work?

[ikolev21]

I tried that by accident, when I upgraded from USD 20.02 to 21.02, hoping it might solve the problem. C4D uses USD 20.02, so while we used the same version both libraries indeed had the same namespace. But when I upgraded to 21.02 our namespace changed (as the namespace includes the version).

The DYLD callback filters the constructor/destructor functions from the loading library by their section tag, not by their namespace. And the section tag is constant, doesn't include the version.

But I'm afraid changing the section tag didn't solve all problems. Both libraries now load without errors, but I get errors on usage. These might be related to improper loading of the plugInfo.json files, I haven't investigated yet.