Support ld.so $ORIGIN somehow

[paulproteus]

Right now, software that uses the ld.so feature called $ORIGIN seems to break in Sandstorm.

In particular, this feature (as I understand it) refers to the ability to pull in *.so files from the directory tree where a program is installed.

See https://www.technovelty.org/linux/exploring-origin.html for a detailed exploration of the feature.

See https://github.com/pwais/opengrok-spk/commit/ca7ffe03341e516e5511c4b97e9ae680d559db42#commitcomment-12602361 for a discussion of an app on Sandstorm that would benefit from this.

[kentonv]

Is it possible to support this without /proc? It seems likely the answer is "no". At least until we have our own userspace ELF loader.

But it seems like for most Sandstorm apps, setting LD_LIBRARY_PATH should be a fine work-around? Maybe we could automate setting it for Java apps in vagrant-spk?

[paulproteus]

I imagine we can do this with an LD_PRELOAD that wraps fork() or exec(). I haven't dug into that idea. I imagine @geofft knows.

[geofft]

I wouldn't go the LD_PRELOAD route unless you already have one on every process, although it would work. For this particular case, adding the hard-coded Java directory seems like a better solution. Alternatively, something (vagrant-spk?) could examine binaries to see if they use $ORIGIN, canonicalize that to the current path, and chrpath them. It's not like binaries are going to move around once they're inside a Sandstorm package, so freezing their current location into the rpath is fine.

I don't think a custom ELF loader helps you, because by that time, you've already lost knowledge of what your executable's path is, and the only way to get that is /proc/self/exe.

[geofft]

Come to think of it, since you have your own root filesystem, you can list /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/jli in /etc/ld.so.conf.d (then run ldconfig). Simpler than exporting LD_LIBRARY_PATH, since some programs might unset it.

I guess the reason Debian doesn't do this is that if you have multiple Java installations, they'll want their own libjli.so. Unclear if that matters for the Sandstorm use case.

[paulproteus]

Also, if we have our own root filesystem, we could look for every ELF file that uses $ORIGIN and chrpath it, right?

[psychemedia]

I've been trying to create OpenRefine as a package and have run up against Java problems, including an inability to locate libjli.so. I'm not well versed enough to be able to figure out what next, having tracked part of the issue that far. (Indeed, I think java permissions/discovery issues may well be breaking stuff earlier, too.)

ldd $JAVA_HOME/bin/java
    linux-vdso.so.1 (0x00007ffd63bf8000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa2b58f5000)
    libjli.so => not found
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa2b56f1000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa2b5346000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fa2b5b12000)

[kentonv]

@psychemedia Have you tried setting LD_LIBRARY_PATH to the directory that contains libjli.so? That should be a reasonable work-around.

[psychemedia]

@kentonv That was part of the fix:

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 
export LD_LIBRARY_PATH="$JAVA_HOME/jre/lib:$JAVA_HOME/jre/lib/amd64:$JAVA_HOME/jre/lib/amd64/jli:$JAVA_HOME/jre/lib/server" 

though in my specific example there were other issues, as resolved vt Drew Fisher here.