Standalone neko app on Linux

[mockey]

A standalone neko app created with nekotools boot doesn't find the necessary libneko.so even when it's in the same folder. This works without problems in Windows (don't know about OSX). By default Linux searches shared libraries only in standard lib-paths or in the LD_LIBRARY_PATH env var. The easiest fix for this seems to be adding some rpath info to the linker when compiling as described here: http://blog.lxgcc.net/?tag=dt_runpath. I tried it and it worked in my tests. Shall I create a PR?

[ashes999]

+1 this is a key issue for me right now and I would also like to see this resolved

[ncannasse]

This is not an neko issue but a Linux one. libneko.so is loaded by the system and we can't run any code until it's loaded. While Windows allows DLL loading in the same directory as the executable, Linux does not (for security reasons I guess).

In order to be able to do that, use a shell script that does for instance:

#!/bin/sh
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./ mycommand $*

[ncannasse]

Another option is to use xcross which creates statically linked binaries which requires no libneko.so or ndll, but it's not been updated for some time now so you might have issues.

[mockey]

@ncannasse I know it's a Linux issue. I created a pull request that fixes it. It's described here: http://blog.lxgcc.net/?tag=dt_runpath

[ncannasse]

@mockey yes, thank you for the link, but I don't think we should allow this for all of our neko executables by default, given the security issue it might cause. You can simply compile your own neko binary with this flag and it will be picked up and used by nekotools boot

[mockey]

As far as I see this is no security issue but an option explicitly offered by ld-linux for ELF.

http://linux.die.net/man/8/ld-linux:

ld.so understands the string $ORIGIN (or equivalently ${ORIGIN}) in an rpath specification (DT_RPATH or DT_RUNPATH) to mean the directory containing the application executable. Thus, an application located in somedir/app could be compiled with gcc -Wl,-rpath,'$ORIGIN/../lib' so that it finds an associated shared library in somedir/lib no matter where somedir is located in the directory hierarchy. This facilitates the creation of "turn-key" applications that do not need to be installed into special directories, but can instead be unpacked into any directory and still find their own shared libraries.

This seems quite useful for neko to me, especially because it replicates the Windows behaviour.

Using LD_LIBRARY_PATH on the other hand seems to be considered a hack: http://prefetch.net/articles/linkers.badldlibrary.html http://xahlee.info/UnixResource_dir/_/ldpath.html

[ncannasse]

Uhm, actually we already allow loading ndlls from the local path, so I guess we could allow loading .so as well too...

[ncannasse]

But then, do we have a similar option with OSX toolchain ?

[mockey]

Uhm, actually we already allow loading ndlls from the local path, so I guess we could allow loading .so as well too...

For distribution of small apps it absolutely makes sense IMO.

But then, do we have a similar option with OSX toolchain ?

Good question. I'll have a look at OSX tomorrow.

[mockey]

As far as I see OSX behaves similar to Windows. Putting the libneko.dylib in the same folder as the executable works already.

[jgranick]

We've long had an issue with wanting to use Neko for a cross-desktop build target, but unfortunately, on some systems (perhaps it was only Linux, all along) it would not pick up a local copy of libneko and other dependencies, and inevitably require the user to have Neko installed -- defeating a lot of the purpose for distribution.

There is the "xcross" method, but this would require all NDLLs to be statically linked, so we need to rebuild Neko for a new Lime, and oops, for a native extension, so on.

Is this the only patch we would need? If so, this could be very exciting. Thank you so much for taking a look at it :smile:

[mockey]

@jgranick

We've long had an issue with wanting to use Neko for a cross-desktop build target, but unfortunately, on some systems (perhaps it was only Linux, all along) it would not pick up a local copy of libneko and other dependencies

It should have been only Linux and only libneko.so. As far as I see Windows and OSX have no problem with loading neko.dll/libneko.dylib from the folder of the exe.

There is the "xcross" method, but this would require all NDLLs to be statically linked, so we need to rebuild Neko for a new Lime, and oops, for a native extension, so on.

I think xcross only loads a fixed set of static libs as of now. Might be possible to change that, so that it loads external ndlls as well...

Is this the only patch we would need?

Yes, you only need to compile the linux binary with make os=linux.

[ncannasse]

@mockey ok, let's do this : could you test + submit a PR ?

[mockey]

Already submitted: #57. I tested it on a Linux-VM with no neko installed. libgc.so is not needed, is it? I added a var for the linker options to the makefile. It only makes sense for Linux. Not sure how you want to handle that. make os=linux?

[jgranick]

On that note, it would be awesome to be able to use Neko on systems without additional dependencies, either including a libgc binary with Neko or statically linked. So long as Neko can be wrapped up in a single folder for a standalone application, I think we'll have instant "target all desktop" builds working, which is a huge plus for the Neko target

(so this might already be the case, just cheering from the sidelines, I suppose :wink:)

[ncannasse]

@mockey you can check the required dependencies of libneko.so with "ldd" command

[mockey]

libgc.so is a dependency according to ldd. I didn't get any loading error in my tests, though. Maybe some gc-stuff has to be triggered for that. Maybe it's possible to add libgc to the exe folder as well. I'll check...

[mockey]

Hi.

On Mon, 23 Mar 2015 14:33:14 -0700 "Ashiq A." notifications@github.com wrote:

@mockey did you check?

Not yet. Did you? You had a real app running without any problems with the patched neko, right? Did you add libgc in the directory or was it installed on the machine?

Mockey

[ashes999]

@mockey that machine is long gone (Linux VM with live CD). I'll have to re-create it and check. I have to re-download your patched version of neko, because I re-imaged my dev machines.

[ashes999]

@mockey I tested this again. I didn't install libgc; it was installed on the machine already (Ubuntu 14.04 live CD from ISO) in /usr/lib/x86_64-linux-gnu/libgc.so.1 (and libgc.so.1.0.3).

[mockey]

Indeed, it seems to be installed by default on Ubuntu 14.04 at least. Apparently it's used by Mono and by some other apps (e.g. Inkscape) as well. I'll try to do some checks tomorrow, like putting it into the app folder and unistalling it. Maybe you could try that, too, if your VM is stll alive.

[ashes999]

@mockey I'm not sure how to remove the built-in libgc from Ubuntu. (It's also installed by default on 12.04.)

Moving the libgc so files into the game directory doesn't work; I get an error:

error while loading shared libraries: ligc.so.1: cannot open shared object file: No such file or directory

I would guess it's still looking at the old directory/path for the file.

[jgranick]

Libgc would need to be statically linked somehow in order to remove the dependency

[mockey]

OK, I finally took some time to check this. libgc is not loaded directly by neko but by libneko. So neko finds libneko but libneko doesn't find libgc (if it's not installed). But it's possible to add the same linker flag to libneko (rpath: $ORIGIN), then it works. Meaning you can put a neko executable (e.g. created by nekotools), libneko, libgc and std.ndll (and maybe other ndlls) together in a directory and it will run. Would that be useful? I can add that to my patch. Not sure it's worthwhile, though, because libgc seems to be installed by default nowadays (mainly because it's a dependency of Mono, I guess). But it won't hurt otherwise. Also maybe it would be better to define some subdirectory (e.g. ./libs) in the app folder for all the libs. Looks a bit cleaner for distribution...

[ashes999]

@jgranick @mockey do we need libgc? At least on Ubuntu, it's used by a number of applications (Star Office, Firefox, etc.) and removing it actually rendered my system unusable. I think it's a fairly safe bet, at least on Ubuntu, that it's installed already; I don't know about other *nix distros, though. Would it be similar? Is this something we should work around by including libgc anyway?