search

FPGAwars / icestudio

:snowflake: Visual editor for open FPGA boards
https://icestudio.io
GNU General Public License v2.0
1.67k stars 242 forks source link

AppImage has wrong magic header #750

Open SuperSandro2000 opened 1 month ago

SuperSandro2000 commented 1 month ago

Ref https://github.com/NixOS/nixpkgs/issues/316579

A correct AppImage file should have a magic header like

 ▶ LC_ALL=C readelf -h Bitwarden-Connector-2024.3.2-x86_64.AppImage
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 41 49 02 00 00 00 00 00

as described in https://github.com/AppImageCommunity/libappimage/blob/ca8d4b53bed5cbc0f3d0398e30806e0d3adeaaab/src/libappimage/utils/MagicBytesChecker.cpp#L45-L63

icestudio does not have that

 ▶ LC_ALL=C readelf -h ~/icestudio-0.11-linux64.AppImage
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00

grunt-appimage generates the appimage files incorrect, so other tools like binfmt on NixOS or appimage-run cannot detect that. Even the test files in https://github.com/Jesus89/grunt-appimage/blob/master/res/AppImage.32bit have the wrong header.

 ▶ LC_ALL=C readelf -h res/AppImage.64bit
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
cavearr commented 1 month ago

Thanks a lot Sandro, i'll check it for next binaries, thanks again.

crabdancing commented 1 month ago

I'd really appreciate fixed binaries. My SO has been wanting to do some Ice Studio stuff on her NixOS machine for awhile. Thanks so much for looking into this! :)

cavearr commented 4 weeks ago

Hi! @crabdancing and @SuperSandro2000, first of all thanks to both a lot for report this bug.

I renew the AppImage build system and i think all its fine now! you could download the latest AppImage from https://downloads.icestudio.io

If you try and give me feedback about if it works, i'm appreciate very much.

thanks!

NicoleKai commented 4 weeks ago

Hey, I tried multiple versions and none of them worked:

Earlier version gives me this, indicating that the binfmt feature isn't identifying the AppImage (wrong magic numbers):

🐟 ❯ ./icestudio-0.11-linux64.AppImage                                                         ❌127 2024-06-04 (Jun, Tue) 12:29:55
Could not start dynamically linked executable: ./icestudio-0.11-linux64.AppImage
NixOS cannot run dynamically linked executables intended for generic
linux environments out of the box. For more information, see:
https://nix.dev/permalink/stub-ld

Later version gives me:

🐟 ❯ ./icestudio-0.11.3w202406041006-linux64.AppImage                                          ❌127 2024-06-04 (Jun, Tue) 12:29:50
icestudio-0.11.3w202406041006-linux64.AppImage installed in /home/nikoru/.cache/appimage-run/0dc0878b0e35b11a5eebc80ba4c5b7800c498e0fefadcc2a87d68b6d4464baab
installed: X-AppImage-BuildId=0.11.3w202406041006 image: X-AppImage-BuildId=0.11.3w202406041006
/home/nikoru/.cache/appimage-run/0dc0878b0e35b11a5eebc80ba4c5b7800c498e0fefadcc2a87d68b6d4464baab/AppRun: line 58: /home/nikoru/.cache/appimage-run/0dc0878b0e35b11a5eebc80ba4c5b7800c498e0fefadcc2a87d68b6d4464baab/usr/bin/icestudio: No such file or directory

Which is an improvement, I guess. But if I run other AppImages, e.g., Discord, it works fine, so I'm fairly sure it's not my system?

cavearr commented 4 weeks ago

Hi @NicoleKai what linux distribution are you using? thanks for the feedback!

cavearr commented 4 weeks ago

Excuse me! there is an error!! give me some time and i'm telling you!

cavearr commented 4 weeks ago

Hi @NicoleKai ! could you try the last wip? thanks!

SuperSandro2000 commented 4 weeks ago

Hi @NicoleKai what linux distribution are you using? thanks for the feedback!

NixOS like me :)

SuperSandro2000 commented 4 weeks ago

I've noticed a few other tiny things:

and then it coredumps with 

                Stack trace of thread 331534:
                #0  0x00007f3adb735b4a memset (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x22b4a)
                #1  0x00007f3adb71a29e _dl_map_object_from_fd (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x729e)
                #2  0x00007f3adb71b8b1 _dl_map_object (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x88b1)
                #3  0x00007f3adb715835 openaux (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x2835)
                #4  0x00007f3adb7144e1 _dl_catch_exception (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x14e1)
                #5  0x00007f3adb715bd2 _dl_map_object_deps (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x2bd2)
                #6  0x00007f3adb731a03 dl_main (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x1ea03)
                #7  0x00007f3adb72e5f3 _dl_sysdep_start (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x1b5f3)
                #8  0x00007f3adb72fd8e _dl_start (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x1cd8e)
                #9  0x00007f3adb72ebd8 _start (/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2 + 0x1bbd8)
                ELF object binary architecture: AMD x86-64

which could be because of the bwrap inside appimage-run but I am not sure.

When I start it with sudo, the program now opens.

cavearr commented 4 weeks ago

i'll try nixOS i debug in it your architecture is x86 or arm?

cavearr commented 4 weeks ago

Hi Sandro, i think the problem is that tha AppRun that we use is very old (in that time the developer takes an electron app example).

our AppRun try to install mime types asociated to the appImage and execute commands like desktop-file-install, xdg-icon-resource, xdg-mime, y xdg-desktop-menu and some of this commands or operations maybe not exists or need root privileges.

I don't know if you could redirect me to some documentation about how integrate AppImage into OS (file mime asociations for example).

The previous packaging method was did it a long time ago and now i'm rewritting it and take my first contact with AppImage internals, thanks a lot for your help and experience.

crabdancing commented 4 weeks ago

i'll try nixOS i debug in it your architecture is x86 or arm?

I had the same problem as NicoleKai. My system is x86_64-linux.

cavearr commented 4 weeks ago

tomorrow i'll have a new version to fix this.

Thanks to both!

crabdancing commented 4 weeks ago

Awesome. Thank you for your help, @cavearr :)

cavearr commented 4 weeks ago

Hello my nixos friends! I'm doing my research, I don't know Nixos very well, but with this I'm delving into it and it's amazing! thank you so much!

Otherwise, I need to explore how to package our icestudio AppImage in Nix form and will probably explore how to upload it to the nixo package store for easier installation.

In the meantime I explore this and learn how to do it (the problem is with the nwjs core which uses external dependencies and this is a problem in nixos). I have been able to run it without problems and without sudo with appimage-run

I install the package as a normal user:

nix-env -iA nixos.appimage-run

and then run icestudio:

appimage-run icestudio-XXXXXXXXX-linux64.AppImage

where XXXXXXXX is the version number

If you could try it and confirm that this way it works on NixOS for the moment while I work to get a better solution.

Thank you very much to all

crabdancing commented 4 weeks ago

Actually, the Nix store contains scripts for packaging AppImages as native packages, once they're created. It's not ideal (less package customizability) but it works pretty well. A fair few NixOS packages are actually just pulled from AppImages.

If you can get the AppImage itself working, I would be happy to make a standalone flake for further testing & upstreaming into nixpkgs!

For a little background, from the log you can see that NicoleKai's using the AppImage binfmt feature. Under the hood, this works by configuring the Linux kernel to recognize the magic number of a binary format, and pass it to a shim/handler/interpreter/executor process. E.g., you could use this to run a Java .jar directly, like any normal executable, or to run AARCH64 executables on an x86_64 system without having to explicitly call QEMU.

In the context of AppImage, this passes the executable to appimage-run, which then applies the workarounds to deal with the binary. Thus, a system with:

programs.appimage.enable = true;
programs.appimage.binfmt = true;

treats calling ./icestudio-XXXXXXXXX-linux64.AppImage as the same as saying appimage-run ./icestudio-XXXXXXXXX-linux64.AppImage.

Sadly, I've tried to latest version too, and I got the same error NicoleKai did. (I also tried calling it with appimage-run, just to make sure binfmt call chain wasn't itself somehow breaking it.)

I hope that helps.

Anyways, I would be willing to try again if you wish. Thanks again for looking into this. :)

cavearr commented 4 weeks ago

The latest wip AppImage is only AppImage without zipping it, as you tell me and read in appImage recomendations packaging :)

i have installed nixOS with plasma desktop. How are your environment i need to reproduce your problem to fix it.

Are you sure you try the latest wip from https://downloads.icestudio.io should be an AppImage , not a zip file.

tell me how is your environment configured to clone it and reproduce your fail.

And thank you to you to give me your feedback, i'm very interested icestudio works well in all environmnets without friction.

crabdancing commented 4 weeks ago

All the AppImage files I see give me a zip file to unpack, so I'm not sure what's going on there. I see a few files that show up as '.error', so there might be something going wrong in the build process?

Unfortunately, my system config is a little complicated (system flake spread across many different git repos on my private VCS) so it's kinda hard to share the whole thing. I can tell you that I'm running Plasma 6 on x86_64-linux 6.8.11 kernel:

uname -a
Linux <redacted> 6.8.11 #1-NixOS SMP PREEMPT_DYNAMIC Sat May 25 14:28:41 UTC 2024 x86_64 GNU/Linux

My default shell is Fish. If you're having trouble reproducing the issue, I could try to throw together a minimal configuration.nix or flake.nix to help.

cavearr commented 4 weeks ago

i'm checking it

cavearr commented 3 weeks ago

Sorry for the inconvenience. Github actions generate automatically the zip file from AppImage at download , trying to force appears work but fails at the end.. We are moving outside this but for the moment the download page depends on it.

cavearr commented 3 weeks ago

Hi again! If it's not a nuisance, could you try 0.11.3w202406050906 wip from https://downloads.icestudio.io/

with appimage-run for the moment and not sudo, and if it crash, could you run:

strace appimage-run icestudio....... and send me the output?

Thanks!

NicoleKai commented 3 weeks ago

It works! (on my nix machine) Great job :)

cavearr commented 3 weeks ago

Thanks! i'll try to reduce friction much more.

Thanks again for your feedback!

crabdancing commented 3 weeks ago

I can confirm it works on mine too. Thanks for your help, @cavearr

Once it's integrated into Github CI, I'll prolly write a flake packaging it for Nix once I get some free time!

cavearr commented 3 weeks ago

Thanks to you! when i have the nix package ready i'm telling you in this thread.

jleightcap commented 5 days ago

@crabdancing @cavearr hello! we (a group of Summer of Nix participants) are looking to help package icestudio! if it's possible to collaborate in packaging the now working Appimage, let us know what progress has been made, and we can help pick up from there.

crabdancing commented 5 days ago

I don't think any progress has been made on my end, aside from several long-abandoned attempts from before the AppImage format was actually valid (I don't even remember where I put them). I have a fair bit of experience with Nix packaging, and now that the AppImage format is correct, my guess is that this package is gonna be pretty easy to package. ^w^

crabdancing commented 5 days ago

Since nixpkgs already contains tools for working with v2 AppImages, this should be easy. My recommended starting point would be to look at a simple AppImage-based nixpkgs application.

If you want to go with an explicit extraction-based approach, looking at something like freetube might help. Otherwise, if you want to go with the simplest possible approach and don't need much control, httpie-desktop may be instructive.

crabdancing commented 5 days ago

I was curious about exactly how difficult it would be to do, so I went ahead and sketched out a package for it myself. The Nix flake can be found here, and the pkg.nix is written in the same style as a nixpkgs expression. As expected, it got as far as 'working GUI' on the first try -- though I haven't tried programming anything yet. I will note the following issues:

1) The entrypoint still offers to integrate the AppImage with your system, not realizing that this should not be its job in this context (it should be the package manager's job). 2) The icon is a simple green circle (is this a mistake? I also saw it on the raw .AppImage itself, in Dolphin file browser). 3) This approach seems to provide no share/ directory in the result, which means it won't properly integrate into a GUI as-is. Might want to experiment with the explicit extraction based approach to get ahold of the .desktop files & icons? Not sure.

I would also guess there are probably environment issues I haven't found yet, but they're probably an easy fix too.

Anyways, that's a reasonable starting point for packaging this, I think. Good luck! :3

cavearr commented 5 days ago

Hi my friends! sorry for my delay, i'm busy closing other prioritize tasks.

Today i reopen this, i'm telling you with your advances or tests . All help or feedback is very welcome!!

@jleightcap ,i'm not an expert on nix, in fact i'm learning now thanks to this issue (and is amazing!) all of your help about the nix creation could be cruzial.

I'm telling you in the day and if you think how could we organized better to do this, tell me please.