mdear commented 1 year ago

After an hour of trying, I couldn't figure out how to connect to Libera.Chat for the very first time and was unable to register a username/password so my fallback is to post an issue here.

I want to use linuxdeploy in a cross-compilation environment where my build server is a Ubuntu 20.04.5 LTS and my target server is a CCLinux 3.x (crosscontrol.com v700 / v1200) which is a variant build by CrossControl on top of Yocto Poky Kirkstone Linux.

Linux v700 5.4.47-2.2.0+g5ec03d06f54e #1 SMP PREEMPT Tue Feb 23 11:36:20 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

I'm also approaching CrossControl support with this question.

I'd like to be able to create AppImages so I can do a single-file deploy without having to try to figure out installing all the dependencies an app may have.

I'm successfully using your tool to deploy AppImages to another target, this time a Linux Mint Victoria server (based on Ubuntu 22.04.2 LTS Jammy Jellyfish) but since these are both x86_64 platforms it's an easier job since there is no cross-compilation needed.

My build VM

ccs@ccs-VirtualBox:~/Qt/Projects/remotedesktop$ uname -a Linux ccs-VirtualBox 5.15.0-60-generic #66~20.04.1-Ubuntu SMP Wed Jan 25 09:41:30 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

My Mint target

ccs@ccs-VirtualBox:~/Qt/Projects/remotedesktop$ ssh vcu vcuadmin@vcuproject2030:~$ uname -a Linux vcuproject2030 5.15.0-76-generic #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux vcuadmin@vcuproject2030:~$ ^C

My CrossControl v700 target based on https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-8-applications-processors:IMX8-SERIES

Linux v700 5.4.47-2.2.0+g5ec03d06f54e #1 SMP PREEMPT Tue Feb 23 11:36:20 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

ccs@v700:~/Downloads$ cat /proc/cpuinfo processor : 0 BogoMIPS : 16.00 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd04 CPU revision : 2

processor : 1 BogoMIPS : 16.00 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd04 CPU revision : 2

NeroBurner commented 1 year ago

not a duplicate, but related to https://github.com/linuxdeploy/linuxdeploy/issues/175

TheAssassin commented 11 months ago

I gave up my efforts on cross compiling a while ago as using QEMU and Docker produce less of a headache usually.

Please post a log. I suspect that to make your use case work, you'd need to be able to specify a compatible ldd binary.

An ARM version of linuxdeploy will be available soon, though.

mdear commented 11 months ago

I'm cross-compiling on an x86_64 build VM for an i.MX8 cortexa-35-poky-linux platform.

I've decided that AppImage for production images makes sense for our project and I'm looking for support for this use case.

Our production environment is simple and our boards (CrossControl v700/v1200) are resource-constrained and I have no plans to run Docker or QEMU emulation as speed really matters (automotive application requiring real-time responsiveness).

mdear commented 11 months ago

I've also posted this question in the AppImage forum, FYI :

https://github.com/orgs/AppImage/discussions/1303

As AppImage is, as I understand it, a dependency of linuxdeploy I imagine this support must be extended by them as well.

Any idea on the roadmap for this feature? Any workarounds to getting this working roughly? I just finished cross-compiling a test executable and I'm eager to package it and try it out on our target.

CrossControl says they don't have a version of ldd that runs on x86_64 against a i.MX 8 ARM binary/shared library. Next step is for me to take a look at more commonly available cross-compilation toolchains. PS. I already have some of the "ldd" functionality using "readelf -d " or "objjdump -p " and I grep for "NEEDED" to display missing libraries.

TheAssassin commented 11 months ago

I just looked up your platform. It's just another ARMv8 (aarch64) board. All the AppImage tools that provide such builds should run fine there, there is no reason why not.

At the moment, cross compilation is not supported in linuxdeploy, though. You can run the tool in a chroot, Docker container or whatever other QEMU binfmt_misc enabled environment. A "cross build" mode is not available; you'd need to convince the tool of running some ldd equivalent that traces the libraries properly. Same goes for all the input plugins (e.g., linuxdeploy-plugin-qt). Contributions welcome. As a workaround, you might run just linuxdeploy itself with QEMU in a suitable Docker container.

Building the final AppImage can be done on foreign architectures easily. appimagetool and also linuxdeploy-plugin-appimage (which uses the former internally) both accept a runtime parameter in which you can specify any runtime for any architecture. This means you can use, e.g., linuxdeploy-plugin-appimage for x86_64 with an aarch64 AppImage runtime.

As AppImage is, as I understand it, a dependency of linuxdeploy I imagine this support must be extended by them as well.

I also maintain most of AppImage toolchain, so we can collaborate on adding whatever is necessary. On the AppImage side, I suppose no work needs to be done. You should test some generic AArch64 AppImage on your boards, though, just to make sure the runtime itself works.

TheAssassin commented 11 months ago

Given you run on some embedded platform, do you even need the single-file-bundle feature? Terminology wise, we need to separate AppImages from their payload, the so called AppDirs. AppDirs are basically portable bundles. All the AppImage format adds to that is that they are then shipped as single files.

linuxdeploy calls itself an AppDir build tool, i.e., it creates those portable AppDir bundles. AppImage is just one of the possible output formats.

I'm not saying you shouldn't use AppImages, though. What a use case! AppImages running in cars!

mdear commented 11 months ago

lol! I really appreciate the replies, @TheAssassin . I'm a new transplant from spending decades in the telecommunications industry (where single-file deployment is commonplace for networking devices such as switches and routers), to the embedded vehicle space. I seek to do most of the hard work once, on a cross-compilation build machine, and only then push out single-file deployments to multiple targets. The idea is to make the deployments cookie-cutter and to remove the possibility of human error. As long as the latency of AppImage unpacking is still reasonably time-efficient (as cold reboot to in-service latency is important to keep as low as possible) then I think there are advantages in using this strategy.

I'm more than happy to contribute pull requests, I just need a pointer to get started (I imagine I wouldn't run linuxdeploy and its' plugins and appimage as AppImage binaries but probably in a developers mode). I'm thinking of using as a workaround "readelf -d | grep NEEDED" or "objdump -p " run on x86_64 cross compilation VM to suss out the dependency web, since I don't appear to have a version of "ldd" that runs on x86_64 against an ARM binary.

PS. I've also posted on the NXP community to see if a version of "ldd" can be made available:

https://community.nxp.com/t5/i-MX-Processors/Looking-for-quot-ldd-quot-that-runs-on-x86-64-platform-against/m-p/1773463#M217246

mdear commented 11 months ago

I'm also expecting that by running my AppImage on my ARM platform that it may unpack its embedded file system to a well-known persistent location (other than /tmp) so that subsequent runs are more performant. I imagine it may be possible to configure this kind of deployment.

TheAssassin commented 11 months ago

The dependency lookup algorithm based on ldd is very much limited, but as long as the tool is used in the classic scenario, it works exceptionally well. We can (and probably should) replace this eventually. ldd is relatively foolproof since it asks the loader what it needs to load, after all; a readelf approach does not compare symbols for instance and will be less reliable.

I think the strategy here must be to either provide an ldd (output) compatible script for your specific use case or implement alternative methods in the C++ code. The former seems more straightforward at first; a simple shell or Python script that prints output similar to ldd should actually do the trick along with an environment variable LINUXDEPLOY_LDD (or similar) to specify the path (honestly, at first, simply setting $PATH to include that tool should work just fine).

I'm also expecting that by running my AppImage on my ARM platform that it may unpack its embedded file system to a well-known persistent location (other than /tmp) so that subsequent runs are more performant

There is a NO_CLEANUP environment variable in the runtime that pretty much prevents the files from being cleaned up; upon successive runs, the AppImage should not reextract fles that exist. It's a little prone to errors (most commonly when running more than one instance in parallel) and could use some love. I encourage you to experiment with that. Contributions welcome there as well.

mdear commented 11 months ago

I may also consider running QEMU on my build VM to do the ARM packaging, but I'm also trying to simplify the toolchain as much as possible, so will consider less complex alternatives.

mdear commented 11 months ago

Also, the guys over at CrossControl have promised to send a script that emulates ldd output on x86_64 against ARM binaries, I'll take a look at it and see if it's reasonably designed, and if so, will post it here (with their permission of course).

mdear commented 11 months ago

Also, to make things more interesting, I'm working in Qt_6.4 as it's a new project with no need for older frameworks (I saw some of your recent modifications in the linuxdeploy-qt plugin for ARMv8 and they appear to be coded against Qt5, I imagine it wouldn't be too difficult for me to extend support)

mdear commented 11 months ago

I should also ask what in your opinion is the easiest way to encode some metadata in an AppImage that indicates a version number, and a way for code to extract that version number from the unpacked file system. This will allow us to keep our AppImages organized and will allow us to show a version number on the GUI.

Likely this would be to introduce a JSON or YAML file in the AppDir structure and give apps a hint as to what directory it ends up being unpacked in, likely via an environment variable. I imagine this would probably be very simple to accomplish.

mdear commented 10 months ago

Ok, circling back to this task, I have an aarch64 QEMU environment ready with sshfs enabled, goal is to package a Qt demo app on an x86_86 build VM to run on a CCLinux v700/v1200 aarch64. Here goes ...

mdear commented 9 months ago

linuxdeploy_strace_Feb9_2024.txt file_aarch64_Qt_lib_Feb10_2024.TXT gtest_all_cc_build_strace_hang_Feb10_2024.TXT

I'm running linuxdeploy but it's failing without giving enough information to debug properly. I have a patch and I need to ask you please to build it for me because my build is hanging.

I've made it far enough to create a Yocto Poky Linux Kirkstone binary with all libraries required to build linuxdeploy, my "cmake ." has passed successfully, but my "cmake --build ." hangs .

Here's my requested diff that I'm requesting you use to generate a linuxdeploy-aarch64_AppImage (Since I can't build or test it, I'm not comfortable submitting this as a pull request):

diff --git a/lib/linuxdeploy-desktopfile b/lib/linuxdeploy-desktopfile
--- a/lib/linuxdeploy-desktopfile
+++ b/lib/linuxdeploy-desktopfile
@@ -1 +1 @@
-Subproject commit aa7423539d6ecdb6eb3bc6c6607247d3f7ff8bb3
+Subproject commit aa7423539d6ecdb6eb3bc6c6607247d3f7ff8bb3-dirty
diff --git a/src/core/elf_file.cpp b/src/core/elf_file.cpp
index f199b21..89101e4 100644
--- a/src/core/elf_file.cpp
+++ b/src/core/elf_file.cpp
@@ -173,7 +173,7 @@ namespace linuxdeploy {
                 ifs.read(magicBytes.data(), 4);

                 if (strncmp(magicBytes.data(), "\177ELF", 4) != 0)
-                    throw ElfFileParseError("Invalid magic bytes in file header");
+                    throw ElfFileParseError("Invalid magic bytes in file header : " + path.string() + magicBytes.data());

                 d = new PrivateData(path);
                 d->readDataUsingElfAPI();

Here are the commands I'm using to try to create my package, and the result I get (the V700 directory sysroot comes from CrossControl, the manufacturer of the v700/v1200 screens which run i.MX8 aarch64 processors, running an OS customized by the manufacturer on top of Yocto Poky Kirkstone, which are the targets that I ultimately am packaging for).

export LD_LIBRARY_PATH=/lib:/usr/lib:/home/ccs/Qt/Projects/CortexBuilds/glibc_2.35_out/lib:/home/ccs/Qt/Projects/CortexBuilds/V700/Qt-6.4.1/lib:/home/ccs/Qt/Projects/CortexBuilds/glibc_2.34_out/lib:/home/ccs/Qt/Projects/CortexBuilds/V700/sysroots/cortexa35-poky-linux/lib:/home/ccs/Qt/Projects/CortexBuilds/V700/sysroots/cortexa35-poky-linux/usr/lib

./linuxdeploy-aarch64.AppImage --desktop-file /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.desktop --appdir /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir --executable /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice --icon-file /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.png --output appimage  --plugin qt --library=/home/ccs/Qt/Projects/CortexBuilds/V700/Qt-6.4.1/lib --library=/home/ccs/Qt/Projects/CortexBuilds/glibc_2.35_out/lib --library=/lib --library=/usr/lib --library /home/ccs/Qt/Projects/CortexBuilds/V700/sysroots/cortexa35-poky-linux/lib --library /home/ccs/Qt/Projects/CortexBuilds/V700/sysroots/cortexa35-poky-linux/usr/lib

linuxdeploy version 1-alpha (git commit ID 2b73a21), GitHub actions build 260 built on 2024-01-09 17:18:12 UTC

-- Creating basic AppDir structure --
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/bin/
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/lib/
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/share/applications/
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/share/icons/hicolor/
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/share/icons/hicolor/16x16/apps/
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/share/icons/hicolor/32x32/apps/
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/share/icons/hicolor/64x64/apps/
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/share/icons/hicolor/128x128/apps/
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/share/icons/hicolor/256x256/apps/
Creating directory /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir/usr/share/icons/hicolor/scalable/apps/

-- Deploying dependencies for existing files in AppDir --

-- Deploying shared libraries --
terminate called after throwing an instance of 'linuxdeploy::core::elf_file::ElfFileParseError'
  what():  Invalid magic bytes in file header
./deploy_dice_app.sh: line 5:   832 Aborted                 ./linuxdeploy-aarch64.AppImage --desktop-file /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.desktop --appdir /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.AppDir --executable /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice --icon-file /home/ccs/Qt/Examples/Qt-6.6.0/demos/build-dice-DOCKER_IMAGE_KTECHNOLOGY_V700_SDK_3_2_0_0_QT_6_4_1_F7D1A901936A-Debug/dice.png --output appimage --plugin qt --library=/home/ccs/Qt/Projects/CortexBuilds/V700/Qt-6.4.1/lib --library=/home/ccs/Qt/Projects/CortexBuilds/glibc_2.35_out/lib --library=/lib --library=/usr/lib --library /home/ccs/Qt/Projects/CortexBuilds/V700/sysroots/cortexa35-poky-linux/lib --library /home/ccs/Qt/Projects/CortexBuilds/V700/sysroots/cortexa35-poky-linux/usr/lib
ccs@ccs-VirtualBox:~/Qt/Projects/CortexBuilds/scripts$

Here's my attempt to build linuxdeploy in my aarch64 emulated environment:

root@qemuarm64:/home/ccs/workspaces/git/linuxdeploy/build# cmake ..
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using ccache
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2")
-- Checking for module 'gtest>=1.11.0'
-- COMMAND /usr/bin/pkg-config  --print-errors;--short-errors;gtest >= 1.11.0
--   No package 'gtest' found
-- [cmake-scripts in linuxdeploy] googletest not found or too old on system, fetching from GitHub
-- Found Python: /usr/bin/python3.10 (found version "3.10.13") found components: Interpreter
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Using ccache
-- [cmake-scripts in linuxdeploy-desktopfile] target gtest found, not setting up again
-- [linuxdeploy-desktopfile] Test test_desktopfile is excluded from ALL, not adding as test
CMake Deprecation Warning at src/core/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- Checking for module 'libpng'
-- COMMAND /usr/bin/pkg-config  --print-errors;--short-errors;--exists;libpng
--   Found libpng, version 1.6.39
-- Checking for module 'libjpeg'
-- COMMAND /usr/bin/pkg-config  --print-errors;--short-errors;--exists;libjpeg
--   Found libjpeg, version 62
-- Found CImg: /usr/include (found version "334")
-- Generating excludelist
-- [generate-excludelist.sh] downloading excludelist from GitHub
wget: note: TLS certificate validation not implemented
sha256sum: excludelist.h: No such file or directory
-- [generate-excludelist.sh] changes detected, updating excludelist.h
-- [linuxdeploy] Adding test test_appdir
-- [linuxdeploy] Adding test test_linuxdeploy
-- [linuxdeploy] Adding test test_elf_file
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ccs/workspaces/git/linuxdeploy/build

root@qemuarm64:/home/ccs/workspaces/git/linuxdeploy/build# cmake --build .
[  1%] Building CXX object _deps/__ld_gtest-build/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o

(hangs)

[linuxdeploy_strace_Feb9_2024.txt](https://github.com/linuxdeploy/linuxdeploy/files/14227714/linuxdeploy_strace_Feb9_2024.txt)

mdear commented 9 months ago

I tried disabling testing and the builds appear to be progressing now, perhaps the builds just happen so slowly in the nested virtualization that they get clobbered by some kind of watchdog.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 14d7fcf..0e1b278 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -48,7 +48,7 @@ include(CTest)

 if(BUILD_TESTING)
     # including this before including lib/ makes sure that the top level project's gtest is used everywhere
-    include(${PROJECT_SOURCE_DIR}/lib/cmake-scripts/include-or-build-gtest.cmake)
+    #include(${PROJECT_SOURCE_DIR}/lib/cmake-scripts/include-or-build-gtest.cmake)
 endif()

 add_subdirectory(lib)
@@ -56,6 +56,6 @@ add_subdirectory(lib)
 add_subdirectory(src)

 if(BUILD_TESTING)
-    enable_testing()
-    add_subdirectory(tests)
+    #enable_testing()
+    #add_subdirectory(tests)
 endif()
diff --git a/lib/linuxdeploy-desktopfile b/lib/linuxdeploy-desktopfile
--- a/lib/linuxdeploy-desktopfile
+++ b/lib/linuxdeploy-desktopfile
@@ -1 +1 @@
-Subproject commit aa7423539d6ecdb6eb3bc6c6607247d3f7ff8bb3
+Subproject commit aa7423539d6ecdb6eb3bc6c6607247d3f7ff8bb3-dirty
diff --git a/src/core/elf_file.cpp b/src/core/elf_file.cpp
index f199b21..89101e4 100644
--- a/src/core/elf_file.cpp
+++ b/src/core/elf_file.cpp
@@ -173,7 +173,7 @@ namespace linuxdeploy {
                 ifs.read(magicBytes.data(), 4);

                 if (strncmp(magicBytes.data(), "\177ELF", 4) != 0)
-                    throw ElfFileParseError("Invalid magic bytes in file header");
+                    throw ElfFileParseError("Invalid magic bytes in file header" + path.string() + magicBytes.data());

                 d = new PrivateData(path);
                 d->readDataUsingElfAPI();

mdear commented 9 months ago

After disabling the Google Test, a few files did compile, but the compilation ultimately hung.

I also tried the "cmake --build ." with a "-j 4" suffix and I just got multiple compilations hanging and failing instead of just one.

gtestDisabledCompileStillHangsFeb11_2024.txt

mdear commented 9 months ago

I gave the QEMU aarch64 ARM machine six CPUs and 8GB of memory (since top was showing default settings were too low), the build appears to be progressing now. More news as it happens.

Update : The build succeeded! I'll now work to produce a pull request that will fix the issue I'm having (linuxdeploy trying to do a "readelf" on a directory).

mdear commented 9 months ago

I was sending in a directory into the "--library" parameter incorrectly, which led to the failure. Is there a way you could warn the user that directories are not allowed, since the error message wasn't explicit, I had to read code and build/debug to figure out what I had done wrong.

https://github.com/linuxdeploy/linuxdeploy/issues/275

mdear commented 9 months ago

Making progress - I ran linuxdeploy and it gathered dependencies as expected.

As the linuxdeploy Qt plugin needs tools from package qtdeclarative-tools, and after building the meta-qt6 layer against branch 6.4.1 I found it failed (according to https://endoflife.date/qt it's end-of-support) so I'm trying to build under Qt 6.5 Long Term Support in the hopes that I'll be able to get a successful build under Yocto Poky Kirkstone branch.

https://forum.qt.io/topic/154563/yocto-build-failed-cannot-resolve-git-codereview-qt-project-org-qt-tqtc-qmlcompilerplus-git

mdear commented 8 months ago

I just wanted everybody to know that I finally succeeded in creating an AppImage that runs on my ARMv8 target, the demo Dice app that comes with Qt 6.5.3. The linuxdeploy/appimagetool toolchain got me 95% of the way there, but I still had to do some manual packaging steps. I've raised issues across multiple repos with my findings and suggestions.

@TheAssassin , thanks for maintaining this excellent toolchain! I'll be continuing to learn more and deep diving on the qt/appimage plugins to better understand them, and will direct any pull request should I become confident enough to generate them.

mdear commented 8 months ago

Ok, I succeeded in creating an AppImage that removes dependencies on enough underlying native libs that it is able to run on multiple underlying Linux deployments (this is a requirement of ours, as we need to be best-effort forward-compatible with current and future OS installations).

On the advice of @TheAssassin I built my own version of Yocto Poky Kirkstone Linux against a qemuarm64 ARMv8 platform.

We rely on CClinux from CrossControl, CClinux2 is built on top of Yocto Poky Dunfell, CClinux3 is built on top of Yocto Poky Kirkstone.

My previous AppImage ran on CClinux3 but not on CClinux2, meaning that I wasn't capturing all the dependencies.

The linuxdeploy toolchain got me around 95% of the way there, the rest I had to do manual file placement (issues raised across the various linuxdeploy repos to track each issue I found).

I used appimagetool manually to do final packaging once I manually got the AppDir tuned properly.

All this effort begs the question whether the linuxdeploy toolchain may benefit from an optional portability packaging mode that ensures best-effort future portability of the AppImage to different underlying OS versions within the same OS family.

Since I had trouble compiling the linuxdeploy-qt plugin, I wasn't able to validate the PR https://github.com/linuxdeploy/linuxdeploy-plugin-qt/pull/157 (I executed these steps manually as described below).

UPDATE I was able to compile the qt plugin after all, please see https://github.com/linuxdeploy/linuxdeploy-plugin-qt/issues/162#issuecomment-1978623169

Link to the final AppImage:

https://www.dropbox.com/scl/fi/b77f7nkaomv9koko7my7s/Dice-aarch64.AppImage?rlkey=u1elt7fdb4534mfo60cjnszcv&dl=0

Summary of changes:

Adjust apprun-hooks/linuxdeploy-plugin-qt-hook.sh with a code block to set needed QT/QML variables. I set LD_LIBRARY_PATH just to ensure that it didn't inherit it from the calling shell (not sure that was necessary, especially since you manually set the rpath of deployed libs to $ORIGIN).

    # Begin modification mdear
    QT_BASE_DIR="$(readlink -f "$(dirname "$0")")"
    export LD_LIBRARY_PATH=${QT_BASE_DIR}/usr/lib:${QT_BASE_DIR}/lib:/lib:/usr/lib
    export QT_PLUGIN_PATH=${QT_BASE_DIR}/usr/plugins
    export QT_INSTALL_PLUGINS=${QT_PLUGIN_PATH}
    export QT_INSTALL_QML=${QT_BASE_DIR}/usr/qml
    export QML2_IMPORT_PATH=${QT_INSTALL_QML}
    export QML_SOURCES_PATHS=${QT_INSTALL_QML}
    export QT_QPA_PLATFORM_PLUGIN_PATH=${QT_PLUGIN_PATH}/platforms
    # End modification mdear

Adjust the linuxdeploy exclusions list to package the gcc libraries (which will most certainly change from underlying release to underlying release). I also packaged two local text/font rendering libs only because they appeared to have been excluded due to inconsistencies in Arch Linux and I knew I was running on Yocto Poky baselines:

    ccs@ccs-VirtualBox:~/workspaces/git/linuxdeploy/src/core$ cat excludelist.h  | grep "//" | grep -v http
    // Including because it's part of glibc, which can change from distro to distro, ensure not to break if underlying OS glibc changes
    //   "ld-linux.so.2",
    //   "ld-linux-x86-64.so.2",
    //   "libanl.so.1",
    //   "libBrokenLocale.so.1",
    //   "libcidn.so.1",
    //   "libc.so.6",
    //   "libdl.so.2",
    // Including because we're not on broken Arch Linux
    //  "libfreetype.so.6",
    // Including because we're not on broken Arch Linux
    //  "libharfbuzz.so.0",
    // Including because it's part of glibc, which can change from distro to distro, ensure not to break if underlying OS glibc changes
    //  "libm.so.6",
    //  "libmvec.so.1",
    //  "libnss_compat.so.2",
    //  "libnss_dns.so.2",
    //  "libnss_files.so.2",
    //  "libnss_hesiod.so.2",
    //  "libnss_nisplus.so.2",
    //  "libnss_nis.so.2",
    // Including because it's part of glibc, which can change from distro to distro, ensure not to break if underlying OS glibc changes
    //  "libpthread.so.0",
    //  "libresolv.so.2",
    //  "librt.so.1",
    //  "libstdc++.so.6",
    //  "libthread_db.so.1",
    //  "libutil.so.1",
    ccs@ccs-VirtualBox:~/workspaces/git/linuxdeploy/src/core$

Here's a list of ARMv8 libraries under Yocto Poky Kirkstone which have their interpreter explicitly set, which indicates that for completeness I perhaps should have packaged and reset the interpreter of these libs (however, just by resetting libc.so.6my Qt QML app ran just fine):

    ccs@qemuarm64:~/workspaces/git/poky/build/tmp/sysroots-components/cortexa57$ find . -name "*so*" -print | xargs file | grep "interpreter /"
    ./glibc/lib/libc.so.6:                                                                                                  ELF 64-bit LSB shared object, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=84d2df611af8616e3fa57490613e41c033aac0d4, for GNU/Linux 3.14.0, with debug_info, not stripped
    ./libcap/usr/lib/libpsx.so.2.66:                                                                                        ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=ea1688b97f0eb63db2758185e2900394aa047cc2, stripped
    ./libcap/lib/libcap.so.2.66:                                                                                            ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=1d20fa7b65a3229a2ba12ff1efa25eeec4de3b88, stripped
    ./glibc-stash-locale/lib/libc.so.6:                                                                                     ELF 64-bit LSB shared object, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=84d2df611af8616e3fa57490613e41c033aac0d4, for GNU/Linux 3.14.0, with debug_info, not stripped
    ccs@qemuarm64:~/workspaces/git/poky/build/tmp/sysroots-components/cortexa57$

I manually installed the following:
- usr/qml/QtWayland
- usr/plugins/wayland-decoration-client
- usr/plugins/wayland-graphics-integration-client
- usr/plugins/wayland-shell-integration
- libqwayland* into usr/plugins/platforms
- libqvnc* into usr/plugins/platforms (just so I could run apps and then connect to them remotely)
- qml/QtSensors/* into usr/qml
- plugins/generic into usr/plugins
- The following into usr/lib:
- libQt6Wayland*
- libQt6Wl*
- libglapi.so.0
- ld-linux-aarch64.so.1 // Ensure this dynamic linker is used rather than the underlying platform's linker
- libdl.so.2 // This one surprised me. I did say I was not a deep Linux / glibc guru, right? Apparently, the system depends on this lib even though it doesn't appear in any "ldd" output (a hidden dependency apparently). It is part of the standard glibc library.
Introduce into AppDir the line "cd $this_dir" after the line where $this_dir is assigned. This was necessary due to the fact that I didn't seem to be able to use the $ORIGIN variable when setting the main application's interpreter via patchelf --set-interpreter.

patchelf --set-rpath '$ORIGIN' usr/lib/libc.so.6
readelf -d usr/lib/libc.so.6 | egrep -i -e '(rpath|runpath|ORIGIN|NEEDED)'
file usr/lib/libc.so.6

patchelf --set-interpreter './usr/lib/ld-linux-aarch64.so.1' usr/bin/dice

I applied the following patch to ensure that the exclusion list wasn't refreshed automatically (since I needed to ensure my manual changes to the exclusion list stuck - not production quality, just enough to move forward):

diff --git a/src/core/generate-excludelist.sh b/src/core/generate-excludelist.sh
index d92cb13..5c4277b 100644
--- a/src/core/generate-excludelist.sh
+++ b/src/core/generate-excludelist.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#/bin/bash

 # Copyright 2018 Alexander Gottwald (https://github.com/ago1024)
 # Copyright 2018 TheAssassin (https://github.com/TheAssassin)
@@ -64,8 +64,9 @@ echo "};" >> "$tempfile"
 # avoid overwriting if the contents have not changed
 # this prevents CMake having to recompile half of linuxdeploy even if nothing changed
 if [ "$(sha256sum $filename | awk '{print $1}')" != "$(sha256sum $tempfile | awk '{print $1}')" ]; then
-    echo "$log_prefix changes detected, updating $filename"
-    cp "$tempfile" "$filename"
+    echo "KTECHNOLOGY : $log_prefix changes detected, not touching $filename to keep local changes intact."
+    #echo "$log_prefix changes detected, updating $filename"
+    #cp "$tempfile" "$filename"
 else
     echo "$log_prefix no changes detected, not touching $filename"
 fi

linuxdeploy / linuxdeploy

Can linuxdeploy support a cross-compilation environment ? #258

My build VM

My Mint target

My CrossControl v700 target based on https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-8-applications-processors:IMX8-SERIES