Call ld-linux relative to cwd; invoke application without launching ld-linux explicitly

[ghost]

Hi,

After the fix in #44 -s deploy does indeed work for qt5 and it does appear to be completely self contained.

However a strange problem has arisen that was not an issue before. The same thing occurs at build 495 and that is why I tested build 497.

My application checks XDG_CONFIG_HOME for the location of the user's config folder.

That is now coming back as: /home/juus/.config/ld-linux-x86-64.so This is also the case where a "My.AppImage.config" folder is available, putting an ld-linux-x86-64.so folder in there too.

Within that .config/ld-linux-x86-64.so folder everything then follows as if it was the normal .config folder, putting a folder with my Appname inside.

I've checked older AppImages (built with linuxdeploy) and that was not a problem, it always returned either /home/juus/.config or My.AppImage.config.

In the log references to "ld-linux-x86-64.so" were:

2020/08/02 11:16:48 Deploying /lib64/ld-linux-x86-64.so.2... ... 2020/08/02 11:16:49 Copying in and patching ELFs which are not already in the AppDir... 2020/08/02 11:16:49 Not writing rpath in commandoo.AppDir/usr/lib64/ld-linux-x86-64.so.2 because itname starts with ld-... 2020/08/02 11:16:49 Patching qt_prfxpath, otherwise can't load platform plugin... 2020/08/02 11:16:49 Offset of qt_prfxpath: 3561388 2020/08/02 11:16:49 Qt prefix directory in the AppDir: commandoo.AppDir/usr/lib/x86_64-linux-gnu/qt 2020/08/02 11:16:49 Relative path from ld-linux to Qt prefix directory in the AppDir: ../usr/lib/x864-linux-gnu/qt5 2020/08/02 11:16:49 Patching qt_prfxpath in libQt5Core.so.5 to .. 2020/08/02 11:16:50 Copying in copyright files...

[probonopd]

https://github.com/Juuliuus/commandoo is what you are trying to package? Can you please post all steps to build and make an AppImage that you used, then I can try to recreate the issue.

I tried https://github.com/probonopd/commandoo/commit/325370f5b7edb9cbb4f2a8d5aa940236ccab903a

but I am getting

Hint: (11030) Start of reading config file /etc/fpc.cfg
Hint: (11031) End of reading config file /etc/fpc.cfg
Free Pascal Compiler version 3.0.2 [2017/02/25] for x86_64
Copyright (c) 1993-2017 by Florian Klaempfl and others
(1002) Target OS: Linux for x86-64
(3104) Compiling lcl.pas
/usr/share/lazarus/1.6.4/lcl/interfaces/lcl.pas(10,3) Fatal: (10022) Can't find unit AllLCLIntfUnits used by LCL
Fatal: (1018) Compilation aborted
Error: /usr/bin/ppcx64 returned an error exitcode
Error: (lazarus) Compile package LCL 1.6.4: stopped with exit code 256
Error: (lazarus) [TLazPackageGraph.CompileRequiredPackages] "Exit code 256"
ERROR: Project dependencies of /home/travis/build/probonopd/commandoo/code/commandoo.lpi
The command "lazbuild -r commandoo.lpi" exited with 2.

Can you show me your code where you are checking XDG_CONFIG_HOME?

[ghost]

Please note that commandoo will not run until libqt5pas is installed (that is the small detail I mentioned in other posts), and I imagine that would also make the appdir build fail if that library is not there. Also note this is all designed for 2020 distros (commandoo was built in Kubuntu 20.04).

Find commandoo.png clean_AppDir.txt (flattened appdir, text only) commandoo-x86_64.AppImage (.config problem) commandoo (standalone executable) commandoo.desktop org.timepirate.commandoo.metainfo.xml

in: https://timepirate.org/downloads/Commandoo.tar.gz

Step 1: ./appimagetool-497-x86_64.AppImage -s deploy ./commandoo.AppDir/usr/share/applications/org.timepirate.commandoo.desktop

Step 2: ./appimagetool-497-x86_64.AppImage ./commandoo.AppDir/

If you have a question regarding the "org.timepirate.commandoo.desktop" filename, this was the only way I could get it all to be error free when checking the xml validity with the gnu online tools. And, anyway, it all works absolutely perfectly...except for this ~/.config problem.

[ghost]

result := GetAppConfigDir( False );//gets home "." location false is xdg .config, true is system /etc

This goes to lazarus: IncludeTrailingPathDelimiter(XdgConfigHome);

which goes to: { Follows base-dir spec, see [http://freedesktop.org/Standards/basedir-spec]. Always ends with PathDelim. } Function XdgConfigHome : String; begin Result:=GetEnvironmentVariable('XDG_CONFIG_HOME'); if (Result='') then Result:=GetHomeDir + '.config/' else Result:=IncludeTrailingPathDelimiter(Result); end;

[ghost]

Hi,

You know I just thought of something. I believe I mentioned my main computer well and truly died a couple days ago. I'm using an older laptop until I get a new main computer, and even though everything is running seemingly good, I'm noticing some small irregularities (behavior of listbox displays, of label caption displays in lazarus), so maybe it has something to do with this computer?

Maybe don't spend too much time on this and I can check again when the new compuer is here (a couple weeks). If you want itt should be simple to reproduce with the files I included the .tar.gz, but now I'm simply not sure that something is messed up on ths temporary replacement computer.

[probonopd]

You know what: I just double-clicked the AppImage from your tar.gz and it immediately worked:

Xubuntu 18.04, no Qt installed, no Lazarus installed.

After running the AppImage, I do have:

me@host:~$ find ~/.config/ld-linux-x86-64.so
/home/me/.config/ld-linux-x86-64.so
/home/me/.config/ld-linux-x86-64.so/commandoo.settings
/home/me/.config/ld-linux-x86-64.so/DBMisc.data
/home/me/.config/ld-linux-x86-64.so/DBCmdLine.data
/home/me/.config/ld-linux-x86-64.so/DBCmd.data
/home/me/.config/ld-linux-x86-64.so/DB.sqlite
/home/me/.config/ld-linux-x86-64.so/searches
(...)

[probonopd]

I think I know what is going on.

Your application correctly gets the path to ~/.config/ (as usual). But I assume that then you want to construct a subdirectory that has the same name as your application. How are you doing this? I suspect you are reading argv[0] or proc/self/exe in some way.

The reason for the strange behavior is that in order to make the AppImage fully standalone, we are shipping a private copy of the ld-linux-x86-64.so loader. And then we are launching your application like this:

<AppImage mountpoint>/lib64/ld-linux-x86-64.so.2 <AppImage mountpoint>/usr/bin/commandoo

You can see it when you run ps ax:

me@host:~$ ps ax
(...)
13086 pts/0    Sl+    0:00 /tmp/.mount_commanK3cvmB/lib64/ld-linux-x86-64.so.2 /tmp/.mount_commanK3cvmB/usr/bin/commandoo
13090 ?        Ssl    0:00 /home/me/Downloads/Commandoo/commandoo-x86_64.AppImage

Hence, when you try to read the name of the running executable using something like argv[0] or proc/self/exe you get ld-linux-x86-64.so instead of commandoo.

Instead, you could just hardcode commandoo and it would immediately work as intended.

But we can also think about whether we can do someting on the AppImage side of things to prevent this from happening. Possibly a solution might be to use https://github.com/AppImageCrafters/AppRun - @azubieta, can you confirm?

When I launch https://transfersh.com/T2cuU/glmark2-latest-x86_64.AppImage and then do ps ax:

13233 ?        R      0:02 /tmp/.mount_glmarkrCsqZm/usr/bin/glmark2 --data-path=/tmp/.mount_glmarkrCsqZm/usr/share/glmark2
13237 ?        Ssl    0:00 /home/me/Downloads/glmark2-latest-x86_64.AppImage

See, no <AppImage mountpoint>/lib64/ld-linux-x86-64.so.2 there. This is of course much better!

So maybe I will switch go-appimage to use https://github.com/AppImageCrafters/AppRun - once I understand it well enough.

[ghost]

Ah, yes I see. Thank you for testing this.

BTW, that is what I meant when I said earlier that it does seem to be completely self-contained. I ran it in OpenSuse which did not work before (linuxdeploy version) because OpenSuse has glibc 2.26 and my app requires 2.29. But the self contained version did, in fact, open and work in OpenSuse. Pretty cool.

[ghost]

I did check. I take exactly what is given to me from the XDG_CONFIG_HOME and that is normally ~/.confg/commandoo

I do not add anything, so the ld-linux bit is replacing 'commandoo', perhaps the inner workings of linux is taking argv[0].

[ghost]

Sorry, apparently I'm tired. You are correct. Lazarus appends "applicationname" as it fetches XDG_CONFIG_HOME.

[probonopd]

I think by using https://github.com/AppImageCrafters/AppRun we can get this fixed but I need some help from @azubieta to show me how to do this. I tried using his AppRun for your commandoo-x86_64.AppImage but I couldn't quite figure out yet what I need to write in .env so that I don't end up with a segfault ;-/

[probonopd]

By the way @Juuliuus your AppImage can even run on Ubuntu 16.04 LTS, just tested it.

[ghost]

Well @probonopd, how cool is that?! I was starting to suspect it might work in older versions after I saw OpenSuse work.

[probonopd]

azubieta confirmed that using https://github.com/AppImageCrafters/AppRun can solve this issue indeed.

[ghost]

Hi @probonopd . There is no urgent need to use the AppImageCrafters tools at this time. I found a simple way to use normal "-s deploy" in go-appimage to work "better" than the AIC AppRun and hook lib (details below) resulting in argv[0] pointing to the payload and not ld-linux.

As I reported at https://discourse.appimage.org/t/is-it-possible-with-s-deploy-to-include-libraries-loaded-at-runtime/1779/36, I was able to make AIC's system work finally by patching the payload to use the copied ld-linux file. But. It gives a really big problem, at least as regards the needs of my program.

A word of explanation: my program is a CLI database =and= testing platform. While the program did finally run on Ubuntu 16, it fails spectacularly at one of its primary aims: testing CLI's. The program worked perfectly in its database functions (libsqlite also bundled inside!). My program depends, heavily, on starting processes and getting the results of stdout/stderr. But even a simple process invoking the "which" command to find the location of the "id" command fails with "which" failing! So the AIC method has somehow made the entire system dependent on my glibc needs?! Ouch.

But taking a bit of the methodology of AIC I was able to modify appdirtool.go to work (including using your original AppRun shell script). The resulting Appimage runs flawlessly on Ubuntu 16, and the March version of OpenSuse (the original problem distro). Both storage and testing in my program work perfectly.

The changes to go-appimage are as follows:

1. generate a new uuid using uuidgen if uuidgen is not available then one must write a "good enough" uuidgen func or bundle it in appimagetool

2. build a unique filename for ld-linux based on payload name and the uuid. I used: /tmp/commando-...uuid...-ld-linux-x86-64.so.2

3. write this ld-linux filename into a file in the AppDir at the same level as AppRun. I used .uuid as a filename, anything will do

4. patch the payload using --set-interpreter switch with the same ld-linux filename

5. modify the AppRun script in two ways: a. get the contents (filename) from the .uuid file b. copy the lib64/ld-linux to that filename in the /tmp folder c. then don't invoke: exec "${LD_LINUX}" --inhibit-cache "${MAIN_BIN}" "$@" but instead the original: exec "${MAIN_BIN}" "$@"

This makes no use of the "-l" switch, it isn't needed.

I can supply you with all the code I wrote, just let me know if you want that and how to go about it.

Does this have side effects? I don't know and I leave that to you with your greater experience. All I know is that commandoo runs perfectly in every system I've tested it on, and the solution, as it is, is quite simple and doesn't depend on anything outside of go-appimage.

One final note: Another testing distro was the nixos distro. AIC's method allows the program to run in nixos, but with the same failure to be not allowed to run my own processes. The modified go-appimage fails in nixos, core dump. A quick glance at strace showed it using a chroot command that I had not seen before, so there is another problem to be solved as regards nixos.

[probonopd]

copy the lib64/ld-linux to that filename in the /tmp folder

Can we find a solution without this step?

I think this is what https://github.com/probonopd/libhookexecv is solving (for Wine).

[azubieta]

So the AIC method has somehow made the entire system dependent on my glibc needs?!

This is provably because the LD_LIBRARY_PATH is being leaked to external process, to avoid this you need to export the following companion variables:

$APPRUN_ORIGINAL_LD_LIBRARY_PATH: original value of the environment variable. Will be restore when executing external binaries. $APPRUN_STARTUP_LD_LIBRARY_PATH: value of the variable when you were starting up your application. Used as reference to detect changes made by the application.

Please check https://github.com/AppImageCrafters/AppRun#libapprun_hooks

5. modify the AppRun script in two ways: a. get the contents (filename) from the .uuid file b. copy the lib64/ld-linux to that filename in the /tmp folder c. then don't invoke: exec "${LD_LINUX}" --inhibit-cache "${MAIN_BIN}" "$@" but instead the original: exec "${MAIN_BIN}" "$@"

Using the embed ld-linux does the trick for running new software in old system, but if fails for the opposite environment. This will remain true as long as we use at least one lib from the system (NVIDIA).

@probonopd Prefixing ld-linux may seem to do the job, but this has a nasty consequence. The /pro/self/exec will point to ld-linux instead of the real application path. Many applications (all the qt ones and the AppImage runtime) depend on this path to know their binary location, those applications will fail.

That's why I opted for patching the PT_INTERPRETER segment on the elf executables.

[ghost]

Hi @azubieta and @probonopd

This is provably because the LD_LIBRARY_PATH is being leaked

You are correct and the two env vars fixed the issue, but with a whole brand new failure.

My program now runs and can execute processes with AIC method =IF= I send them directly to the system through a created process. However, my program also allows piped constructions like "ps -A | grep -i commandoo". When running CLI's of this type I send it out to the shell (bash, zsh, etc). These now fail with core dumps for every command in the pipe except the last.

When I test with my solution those piped commands work properly...

[azubieta]

Great, that's progress! About the new issue, AppRun removes the AppImage private execution environment when an external binary like (/bin/bash) is executed. This also remove the hooks library. If the execution flows returns into the AppDir like in your case where an external bash calls an internal commandoo the private environment doesn't get restored.

The solution will be to also bundle bash, execution ps, grep or other external command should be safe as long as those commands doesn't exec back an internal command.

[probonopd]

That's why I opted for patching the PT_INTERPRETER segment on the elf executables.

Could what https://github.com/probonopd/libhookexecv does be a solution that combines the advantages of both approaches?

[azubieta]

Running binaries using ld-linux <binary> makes proc/self/exe point to ld-linux which breaks all Qt applications and any other that depends on this path.

[ghost]

Hi all,

The solution will be to also bundle bash, execution ps, grep

@azubieta : I bundled bash first alone, then bash, ps, & grep. In all cases the core dump for the elements of the pipes continues to occur.

Even though I tried this just to see =IF= it would work, this solution is no good for my program (or any program that may create a process to the system using pipes). commandoo is a CLI program (the user can run, literally, anything they want) and it should, and it does, allow the user to use whatever shell they want. I simply read the SHELL env var and send piped commands to that. So, for me, and for the tests I've run, the AIC method, elegant though it is, simply is not the answer and becomes prohibitively unfeasible.

Can we find a solution without this step?

@probonopd : I tried a couple variations just to see if they would work: I copied the ld-linux... to the bin folder next to my exec and patched the exec to "./ld-linux..." and "ld-linux...". As I expected (given from what I've read on inet) this did not work: file not found. The only way to get around it that I can see is that we have control over what the mount is named so that we can patch the exec to find ld-linux in the ..../lib64/ folder of the mount. I believe you've already said this isn't possible? If we could control the name of the mount then there would be no need to copy ld-linux, we could just point to the folder containing it.

If the above is not possible, then I'm not sure why you have concern over this copying step. My experience is not as deep in linux as yours, and maybe it has consequences that I'm not yet aware of. But for a couple of reasons I find it a very simple solution to the problem of completely self-contained appimages, and, if nothing else, could be a next stage for go-appimage development.

First: It completely solves the argv[0] problem.

Second: Simple changes only were necessary to your go-appimagetool code, nothing was touched in the methods you use, and, it works in all cases: OpenSuse, Ubuntu 16.04, and even (sort of) in nixos!! (I got it working in nixos, more detail below if you are interested). Calling processes works out of the box. Piped commands work out of the box. What better result can there be for such a simple change? In addition, running multiple copies of the appimage will use the same file because its based on a guid that bundles itself into the appimage. I can't get the resulting appimages to fail in any way, they work perfectly.

Third: there seems to be perfect separation of what my program needs to run (the self containment) and yet the freedom to use the host distro's files to do work ("ps -A | grep xxx").

FWIW, if you are interested in the nixos problem: As I've mentioned before one needs to run the appimage with nixos's "appimage-run". Even though the appimage builds fail in nixos, I noticed that the entire process got far enough along to extract the appimage =and= to copy the ld-linux... file to the /tmp/ folder.

In nixos appimage-run extracts the appimage to: ~/.cache/appimage-run/a big long number/squashfs-root/

Going there I could run commandoo directly from that folder. It ran perfectly! Processes. Piped commands. I could run Geany through commandoo as a child process. And this is despite the fact that all the libraries in that folder are patched by go-appimage, and that the commandoo is patched to use the ld-linux... in the /tmp/ folder (ld-linux needed to be modified to be executable, that didn't get copied over, probably nixos security issue).

The bash commandoo used came from the SHELL env var and was pointed to the peculiar folder that the nixos system uses to keep "snapshots" separate.

It's true that there were no APPDIR or APPIMAGE env variables running it this way, but everything is pointed to the self containment and the separate ld-linux. Because of this my program could not find libsqlite but I allow specifying it and when I pointed it to the libsqlite bundled within, it worked great.

So my feeling at this time is that appimage-run needs an issue for this appimage method, which, after all this is ironed out, I plan to submit. But the resulting appimage is, truly, self contained and, apparently, will work anywhere.

[probonopd]

The only way to get around it that I can see is that we have control over what the mount is named so that we can patch the exec to find ld-linux in the ..../lib64/ folder of the mount.

Do we need this at AppImage creation time or at AppImage run time? At AppImage run time, the mount point is in $APPDIR...

[azubieta]

I've added a new feature to appimage-builder, now it's capable to take an file level snapshot of an application and generate a recipe the includes all the files that were loaded. This allows to build use the tool on non-debian based system.

A similar approach can be used by go-appiamge:

• run the application using LD_DEBUG=libs and strace
• capture both ELF files and data files
• write them down to a list
• build an AppImage from the list

[ghost]

Azu: I've added a new feature to appimage-builder

Cool, I'll look at that after I've got all this current go-appimagetool issue down

Prob: Do we need this at AppImage creation time or at AppImage run time?

Both actually.

Remember the "magic" fix for argv[0] is that the payload (or payloads?) have to be patchelf'd to point to the bundled ld-linux and then the payload is run directly. This patching must be done at build-time, which means we must know the ultimate, mounted ld-linux location (appdir/lib64/ and $APPDIR/lib64/).

But you told me that the mount name is randomized when the AppImage starts up. Then we can not know what the path is until the AppImage starts up and therefore can't patch our payload with a known ld-linux location..

-IF- we can influence the name of the mount, then we could use the $APPDIR/lib64/ path which will then be known to us at build-time since we are influencing the mount name.. This is the best solution for sure, but is it possible? If so, the mount name should be done with a per AppImage GUID so that AppImages don't collide with each other.

If we cannot influence the mount name then there is no other way except to (run-time) copy the ld-linux to a name of our (build-time) choice in the /tmp/ folder (GUID based that travels with the AppImage). This is what I'm currently doing and, as mentioned, it is working everywhere.

if you want to do this all at run-time so that the payload can be patched to look at $APPDIR/lib64/ then patchelf must be included in every appimage, and it must be patched to find a suitable ld-linux. This starts getting circular and creates more prolbems than solutions, no?

[ghost]

Hi @probonopd

Do we need this at AppImage creation time or at AppImage run time?

I'm not sure it you saw the rest of this conversation yet or not. Would you like the code I wrote that addresses this by run-time copying of a build-time determined path? It should be possible to include, along with the original -s and -l switches, yet another switch (-c for complete or copy??) that would encapsulate all of this. Then, in time, it could be decided what works and/or is the most practical?

[probonopd]

Hi @Juuliuus actually this thread has been long both in terms of when it started as well as number of lines to read. Hence I have probably lost track.

Hence, can you please give the "TL;DR:" summary (<= 5 lines) of what you think we should do, and why?

Thanks!

[ghost]

Hi @probonopd (full answer is 3 posts up, Oct 30)

The "magic" fix for argv[0] is that the payload (or payloads?) have to be patchelf'd to point to the bundled ld-linux and then the payload is run directly. This patching must be done at build-time, which means we must know the ultimate, mounted ld-linux location (appdir/lib64/ and $APPDIR/lib64/). Copying at run-time to a filename determined at build-time seems the only option. My opinion? If you want argv[0] fixed, one has no other choice. If it is isolated by another deploy flag (say -c for complete/copy?) then it can be used, or not used, and tested easily.

[probonopd]

Well. The whole situation is not nice.

• Without copying: Even if we made the AppImage mount point fixed and already known at build time, then it would still break as soon as the user would extract the AppImage and run the contents outside of the AppImage, wouldn't it?
• With copying: Could work even in the situation above, but overall sucks

I'm almost thinking that we should not try to fix argv[0] until we have a better solution.

Could a custom ld-linux modify argv[0] in a suitable way? Or a library loaded with LD_PRELOAD?

[ghost]

But you were ok with the -l option using appimagecrafters which is a similar setup, but even more complicated with a settings file, binary AppRun, a special library, and also a copied ld-linux.

The method I have (which works great everywhere but nixos [and even that can be manually fixed]) uses your original apprun shell script with the difference of calling the payload directly, and, yes, a copy of the ld-linux to /tmp/.

• Without copying: I'm not sure I understand. "Without copying" is the "deploy -s" and it works fine except for argv[0].

• With copying: Actually, this might be the problem child. I don't think I tested that, but since the payload is patched to look for ld-linux in /tmp/ that would break it! Good point... That makes me wonder: What is the point of extracting an appimage with the expectation that I can run the payload? I mean, the appimage is built such that it MUST use the bundled libraries, right? And those libraries are patched to work off of ORIGIN??

As to the last two questions:

In my mind argv[0] is non-negotiable. By convention it is the first param of a command line sequence which must be an executable. I'm not familiar with how a custom ld-linux would be made. The keep it as simple as possible philosophy would argue against generating a separate ld-linux I think, but you have more knowledge and experience with that side of things.

And the last: I'm not familiar with LD_PRELOAD. Remember, I didn't even know ld-linux existed until a few months ago... :-O

[probonopd]

But you were ok with the -l option using appimagecrafters which is a similar setup

Yes, but this is go-appimage, and it is designed to be really simple.

[ghost]

In that case, I guess I would say leave argv[0] as it is, pointing to ld-linux. It's not horrible, but it, also, is not terribly pretty. I fixed my program to take that into account, so it doesn't hurt the user other than sorting taskbars: by alpha puts "c"ommandoo down with the "L"s....

[ghost]

Oh, actually. THE answer would simply be: being able to specify the /tmp/ folder mount name. You mentioned to me before that that mount name is random. But is it out of our control? If we can specify the mounted folder name, all the problems go away, and one simply points to the local APPDIR/lib64/...but that would also still break an extracted appimage, yes?

[probonopd]

If we can specify the mounted folder name, all the problems go away

How? What would you do with that information?

[ghost]

Same thing (patchelf the payload so that it can be argv[0]) but this time no need for an external copy of ld-linux.

Just patch to the -mountname-/lib64/....

But again, extracting the appimage with the expectation that I could run the payload by double clicking it would be broken...? Right? I've never extracted an appimage thinking that I would run it. And in this case we're talking about "-s" deploys....

[probonopd]

Yes, being able to run --appimage-extract and then executing the AppRun, and being able to use --appimage-extract-and-run is quite important.

[ghost]

Hmmm. Then I'm thinking its a broken dream. From what I understand ld-linux MUST be called with an absolute path and those paths would then be different for the mounted version vs. the extracted version. And, hence, AppRun would be broken.

[ghost]

Hi @probonopd

So I found a solution that is is pretty simple: rename appdir/lib64/ld-linux-x86-64.so.2 to appdir/lib64/PayloadName, and change AppRun to use that "ld-linux" to run the payload.

Tested on Dev machine & ubuntu16.04 VM

I also tested, in ubuntu 16.04, --appimage-extract and --appimage-extract-and-run, everything runs and argv[0] is "PayloadName".

[azubieta]

rename appdir/lib64/ld-linux-x86-64.so.2 to appdir/lib64/PayloadName

When the system glibc is newer than the one in the AppDir you should use the system ld-linux. Otherwise a crash is expected if some system library gets loaded (which is a requirement to support Nvidia drivers)

[ghost]

Hi @azubieta

When the system glibc is newer than the one in the AppDir you should use the system ld-linux

Ahhh. These are things I'm not familiar with, I must say. I'm focusing on the argv[0] problem. As regards Nvidea, I have no idea, I will leave that for @probonopd to decide/determine. However, I also want to point out that this argv[0] problem is only for the experimental self-contained version of go-appimage. In that case, as far as I know (and we've probably reached my level of behind the scenes interactive linux knowledge), this appimage is required to stay in and use the bundled libraries. If something is going on behind our backs by nvidea, then that becomes a rather ticklish problem...?

Also if the nvidia problem is something that can be addressed, then the decision can still be made if necessary. After all, the re-named to PayloadName file is still, inside, an ld-linux file and can be switched in some manner...?

[ghost]

Hi @probonopd. Is the solution to rename the ld-linux.so to be the same name as the payload acceptable to you? That way when you invoke the "ld-linux" in your AppRun it =is= param[0], and the taskbar and executable name are all corrected. But also see the post immediately above this one. It erases all the complexity that is building around this issue, fixes the issue in a simple manner, and leaves the existing go code almost untouched. It has been tested in all the test beds you mentioned to try with wonderful success. It was a simple answer that was sitting there from the beginning of this issue, and was simply missed because of the preconieved notions that came from elsewhere. I have a new release of my application and I very much want to use this in its AppImage...? One issue with this: there will be two executables in the appdir with the same name, one in \lib64, and one in ...\bin. For that reason I would say a text file should be placed in the \lib64 explaining the situation. In either case, I guess, this thread seems to be stalled, and I would recommend closing it?

[probonopd]

Maybe changing the .interp ELF header in the payload executable is the solution?

https://github.com/orivej/ldcp/blob/2cd63d201d88daad2e878f05f0be728937bdda1a/ldcp.py#L68-L69

Please test https://github.com/orivej/ldcp and see whether that solution suffers from the same issue. If it doesn't, we should look into it.

[probonopd]

Is the solution to rename the ld-linux.so to be the same name as the payload acceptable to you?

No. A payload executable might launch another payload executable from the same AppDir, in which case it'd get really messy really quickly.

[ArcticLampyrid]

Hmmm. Then I'm thinking its a broken dream. From what I understand ld-linux MUST be called with an absolute path and those paths would then be different for the mounted version vs. the extracted version. And, hence, AppRun would be broken.

No, ld-linux can be called relatively, but it's relative to cwd, not $ORIGIN.

See https://github.com/AppImageCrafters/AppRun/blob/master/docs/USAGE.md#make-interpreters-paths-relative

[probonopd]

@ArcticLampyrid if this turns out to work, it's the best application bundle discovery of the decade. No kidding!

[Samueru-sama]

Hmmm. Then I'm thinking its a broken dream. From what I understand ld-linux MUST be called with an absolute path and those paths would then be different for the mounted version vs. the extracted version. And, hence, AppRun would be broken.

No, ld-linux can be called relatively, but it's relative to cwd, not $ORIGIN.

See https://github.com/AppImageCrafters/AppRun/blob/master/docs/USAGE.md#make-interpreters-paths-relative

I would really appreciate an explanation of how to do this exactly.

Tried the example they did with bash with a different binary after deploying everything with go-appimage, it just causes the binary to segfault after running patchelf --set-interpreter.

Anyways, to fix this issue I found this very useful tool which does some tricks with hardlinks.

It also doesn't need to set the rpath of binaries/libraries, since it tells the ld-*.so their location.

[probonopd]

Well, well, well... what we'd actually want is "relative to $ORIGIN", not "relative to cwd"... is this something that would need to be supported in the Linux kernel?

[ArcticLampyrid]

Unfortunately, the kernel currently has no concept of $ORIGIN. The resolution of $ORIGIN is handled by ld.

At present, AppImageCrafters/AppRun employs a trick: it switches the working directory to load the required ld and then injects libapprun_hooks.so via LD_PRELOAD, reverting the working directory back to its original state in the early stages of program startup.

See also:

• https://github.com/AppImageCrafters/AppRun/blob/407700ccef58e1a64cc49856074aee4b1c75d599/src/apprun/runtime_interpreter.c#L220-L221
• https://github.com/AppImageCrafters/AppRun/blob/407700ccef58e1a64cc49856074aee4b1c75d599/src/hooks/exec_utils.c#L103-L113

[probonopd]

Unfortunately, the kernel currently has no concept of $ORIGIN. The resolution of $ORIGIN is handled by ld.

Maybe it would be worthwhile to propose such a concept to the kernel, what do you think?

To patch the Linux kernel to allow for the resolution of $ORIGIN, we would need to modify the kernel's binary loader to handle this feature, along these lines:

• Modify the fs/binfmt_elf.c file, which is responsible for loading ELF binaries, to parse the DT_RPATH and DT_RUNPATH tags in the ELF header. These tags contain the paths that ld uses to resolve shared libraries.
• Add a new function to resolve the $ORIGIN token in the DT_RPATH and DT_RUNPATH tags. This function would need to determine the path of the executable being loaded and replace $ORIGIN with that path.
• Modify the search_binary_handler function to use the new function to resolve the $ORIGIN token when searching for the ld-linux loader.

Roughly like this:

// fs/binfmt_elf.c

// ...

static char *resolve_origin(const char *path, const char *origin)
{
    if (origin == NULL)
        return NULL;

    char *resolved_path = NULL;
    size_t len = strlen(origin) + strlen(path) + 2;
    resolved_path = kmalloc(len, GFP_KERNEL);
    if (resolved_path == NULL)
        return NULL;

    strcpy(resolved_path, origin);
    strcat(resolved_path, path);

    /* Remove any trailing slash.  */
    if (resolved_path[strlen(resolved_path) - 1] == '/')
        resolved_path[strlen(resolved_path) - 1] = '\0';

    /* Remove any .. components.  */
    char *p = resolved_path;
    while (1)
    {
        char *dotdot = strstr(p, "..");
        if (dotdot == NULL)
            break;

        char *prev_slash = strrchr(p, '/');
        if (prev_slash == NULL)
        {
            /* .. at the beginning of the path.  */
            memmove(p, dotdot + 2, strlen(dotdot));
            p += 2;
        }
        else
        {
            memmove(prev_slash, dotdot + 2, strlen(dotdot));
            p = prev_slash;
        }
    }

    return resolved_path;
}

static int load_elf_binary(struct linux_binprm *bprm)
{
    // ...

    /* Find the ld-linux for this ELF executable */
    if (elf_interpreter) {
        char *interp = elf_interpreter;
        char *origin = NULL;

        // Find the path of the executable being loaded
        origin = kmalloc(PATH_MAX, GFP_KERNEL);
        if (origin == NULL)
            return -ENOMEM;
        if (get_task_comm(origin, PATH_MAX, current) < 0) {
            kfree(origin);
            return -EIO;
        }

        // Resolve the $ORIGIN token
        char *resolved_interp = resolve_origin(interp, origin);
        if (resolved_interp == NULL) {
            kfree(origin);
            return -EIO;
        }

        // Search for the ld-linux loader
        elf_interpreter = resolved_interp;
        bprm->interp = load_elf_interp(resolved_interp,
                         &loc->elf_ex,
                         &elf_headers,
                         &interp_load_addr,
                         &interp_map_addr);
        if (bprm->interp == NULL) {
            kfree(origin);
            kfree(resolved_interp);
            return -EIO;
        }

        kfree(origin);
        kfree(resolved_interp);
    }

    // ...
}

Is anyone around who could help to turn this into a working kernel patch and to submit it to the Linux kernel mailing list for review and inclusion in the mainline kernel?

(Llama 3.1 70B was consulted to point me in this direction.)

[probonopd]

Looks like https://github.com/VHSgunzo/sharun and https://crates.io/crates/userland-execve are exactly what we need?

• https://github.com/probonopd/go-appimage/issues/312

So @Samueru-sama, this time for real: if this turns out to work, it's the best application bundle discovery of the decade. No kidding! 👍

[Samueru-sama]

So @Samueru-sama, this time for real: if this turns out to work, it's the best application bundle discovery of the decade. No kidding! 👍

I mean you can see the results and do all kind of needed test with this appimage.

Also this appimage of android tools, which was the reason I switched to sharun since I noticed adb was self reporting as ld-linux.so when I originally tried to bundle all the libs, now it is fixed: