build sys: linux: avoid requiring LD_LIBRARY_PATH for $HFS/dsolib

[mistafunk]

at runtime, our dso is depending on libraries in $HFS/dsolib. as it does not seem to be possible to embed literally "$HFS/dsolib" in the linker RPATH we need to require LD_LIBRARY_PATH before launching houdini which is ugly.

is there a way to avoid LD_LIBRARY_PATH?

while at it, also investigate if there is a reasonable way to limit symbol visibility (i.e. is there a stable set of symbols we can whitelist for palladio so it still loads and runs?)

[mistafunk]

latest info from sidefx support:

Have you tried wrapping $HFS/dsolib in single quotes when passed to rpath?

For example: -Wl,-rpath,'$HFS/dsolib'

I'm not 100% sure that would work but it's worth a try.

And if that fails then the other option is to package boost_filesystem with your HDK plugin and then set your plugin's rpath to '$ORIGIN/.'. That might be a better approach since it is not guaranteed that the boost_filesystem library will exist in any Houdini installation. For example, boost_filesystem no longer exists in any H16.5 build. It's hboost_filesystem in H16.5.

You can take it even further and statically link in boost_filesystem into your plugin. Houdini does not ship with static libraries so you would need to download and build your own boost_filesystem static lib for Boost 1.57.0.

[mistafunk]

maybe best to link statically to our own copy of boost (also allows to use more recent version than sidefx does). on the other hand it goes against best practices to link the same library twice (potential of symbol collision)

[Svenito]

Mixing boost versions from Houdini and system is not a great idea. I have experienced segfaults with smart pointers because of this in 16.5. In 17.0 I am forced to link to both boost and hboost though, where things work a little better. The reason I need both is because my project links to libraries that require regular boost. So it's all getting somewhat messy. Just giving you a heads up on this lovely hboost future :)

You should be able to expand $HFS when setting the rpath though.

[mistafunk]

@Svenito i've tested adding $HFS to the RPATH and this does not work. i believe for security reasons, the linux dynamic linker only knows $ORIGIN as a special value (i.e. it is not an env var, even if it looks like that).

i'm going to try to privately link a static copy of boost to palladio. this should work if we take care to not depend on hboost and make sure the symbol visibility is correctly handled (i.e. not using global variables with "our" boost or just in anon namespaces etc)

[Svenito]

What I meant was that you should know the $HFS value at compile time and so it is expanded when the rpath is set in the compiler args.

[mistafunk]

avoided this problem by switching to internal static boost libraries (thanks to conan for making this a breeze).