Modules cannot be loaded if not in system paths

[mathstuf]

We're packaging ospray as part of ParaView and the libraries that implement backends (e.g., ispc and the ISA libraries) are only searched for in the default system paths. This is not true for ParaView packages. I think the best solution is probably to find out where libospray.so is and search relative to that (either in its directory or, better, a subdirectory since these are ospray-secific "plugins").

This likely won't work for static builds, but maybe that's not a problem.

VTK has code for figuring out the path to the library containing a certain symbol here.

Cc: @demarle

[jeffamstutz]

Yes, we have discussed expanding ospLoadModule() (which is what does the ispc module load in ospInit()) to allow for a prefix path. However, on Unix platforms you can use LD_LIBRARY_PATH to add paths that get searched. Windows would benefit more from specific path variants of ospLoadModule()...

[mathstuf]

On March 28, 2019 8:05:02 PM EDT, Jefferson Amstutz notifications@github.com wrote:

Yes, we have discussed expanding ospLoadModule() (which is what does the ispc module load in ospInit()) to allow for a prefix path. However, on Unix platforms you can use LD_LIBRARY_PATH to add paths that get searched. Windows would benefit more from specific path variants of ospLoadModule()...

Well, I suspect we're actually OK on Linux right now because we manipulate LD_LIBRARY_PATH for other reasons right now (though this won't be the case by the end of the year). I'm seeing it right now on macOS where there is no chance to set DYLD_LIBRARY_PATH for users double clicking on the application.

[jeffamstutz]

Just curious (maybe a question for @demarle?): did this ever work before in your macOS builds, or is this a new problem?

[demarle]

This appears to be new with the bump from 1.6 to 1.8. We are just getting the relocatable packages up now as opposed to testing in the build directory which we did with the bump initially a couple of weeks ago.

[mathstuf]

I think it worked before in that the implementations were likely linked directly rather than loaded at runtime.

I'm going to make a patch which does the equivalent of the vtkResourceFileLocator to get the library's location and find the library relative to that.

I think the proper solution would involve:

• locating them relative to libospray.so
• placing these in a subdirectory (since they're not meant to be linked anyways AFAICT)
• removing SONAME/SOVERSION details (since they're basically plugins)
• make their CMake library type MODULE in shared builds and STATIC in static builds (if supported); libospray.a would link to the impl libraries available.

I'll push my diff doing the first step here, but it's likely to be a bit hacky to get us a viable binary and I don't know if the other points are something you'd think it worth implementing.

[jeffamstutz]

This is fixed in the v1.8.4 release, via PR #312.

[mathstuf]

What are your thoughts on the other bullet points in my previous comments? Should there be a separate issue for that?

[jeffamstutz]

All of those are good suggestions, and we can implement them in v2.0 (especially the STATIC build). We would likely keep modules them as SHARED libraries, as we want to permit users to link against them directly (i.e. add new object types to the ISPC module).

I'd still like to add a version of ospLoadModule() that takes either an assumed full path to the module, or a set of search paths that lets the programmer choose exactly where modules are searched. I would think that's more robust for VTK/ParaView?

We also plan on keeping version information, as this is helpful to understand module compatibility...though maybe that information could be kept internal to the library itself (and not encoding that in the name).

[mathstuf]

Ah, I didn't realize that they were meant to be linked to directly. In that case, the subdirectory probably doesn't make sense.

[mathstuf]

that takes either an assumed full path to the module, or a set of search paths that lets the programmer choose exactly where modules are searched. I would think that's more robust for VTK/ParaView?

Eh. The libraries are built as part of OSPRay, so it should keep track of where its plugins are. VTK/ParaView shouldn't have to know OSPRay's install tree layout to hint paths to it.

[johguenther]

fixed