[appimagetool] allow embedding other architecture runtimes

[probonopd]

Allow embedding other architecture runtimes, e.g., allow a 64-bit appimagetool to create a 32-bit AppImage

[probonopd]

Steps to do:

1. Get AppImageKit to compile on 32-bit CentOS 6 on Travis CI (or at least get runtime.c to cross-compile from 64-bit to 32-bit in the currently working 64-bit Docker image)
2. Make a version of appimagetool that copies that 32-bit runtime instead of the default one into the generated AppImage depending on the architecture of the AppDir that was detected (i.e., if a 32-bit AppDir is detected, then a 32-bit runtime should be embedded) PR welcome!

[jviotti]

Get AppImageKit to compile on 32-bit CentOS 6 on Travis CI (or at least get runtime.c to cross-compile from 64-bit to 32-bit in the currently working 64-bit Docker image)

Did we already do this at some point? See this release for example: https://github.com/probonopd/AppImageKit/releases/tag/6, it has compiled AppImages for both platforms. Did Travis CI upload them?

[jviotti]

Make a version of appimagetool that copies that 32-bit runtime instead of the default one into the generated AppImage depending on the architecture of the AppDir that was detected (i.e., if a 32-bit AppDir is detected, then a 32-bit runtime should be embedded) PR welcome!

So as far as I understand, runtime.c is compiled and stored in the appimagetool binary itself. appimagetool.c knows the positions where this data is stored on itself, and uses such information to inject it in the beginning of the AppImage, right?

If so, what we want is to also include the 32bit version of the runtime in appimagetool, and make appimagetool decide on the fly which to inject, right?

[jviotti]

Also, if we embed both runtimes in appimagetool, then building an 64-bit AppImage from a 32-bit system would also be possible (although probably not a common scenario), at least in theory, or am I missing any other constraint?

[probonopd]

Did we already do this at some point? See this release for example: https://github.com/probonopd/AppImageKit/releases/tag/6, it has compiled AppImages for both platforms. Did Travis CI upload them?

Yes, but that was the old version when we were still using AppImageAssistant. The new one is in the appimagetool/master git branch.

So as far as I understand, runtime.c is compiled and stored in the appimagetool binary itself. appimagetool.c knows the positions where this data is stored on itself, and uses such information to inject it in the beginning of the AppImage, right?

Yes. We could store multiple (additional) runtime binaries for various architectures as normal files inside the appimagetool AppImage though, and use whatever matches the detected architecture of the AppDir about to be packaged.

If so, what we want is to also include the 32bit version of the runtime in appimagetool, and make appimagetool decide on the fly which to inject, right?

Correct, I think that would be the most elegant solution.

Also, if we embed both runtimes in appimagetool, then building an 64-bit AppImage from a 32-bit system would also be possible (although probably not a common scenario), at least in theory, or am I missing any other constraint?

Yes. We might even want to throw in some ARM, too.

[jviotti]

https://github.com/probonopd/AppImageKit/blob/appimagetool/master/.travis.yml#L8

I'm happy to give this a try tomorrow. Any blockers you faced that would be worth mentioning?

[probonopd]

Didn't really try hard, so no.

[fabianloewe]

Why is the runtime embedded in appimagetool at all? Wouldn't it be simpler to just have a copy of the runtime for each processor arch and then choose which should be used?

Yes, you can't use appimagetool as single application anymore but this is why this tool exists: Use the AppImage!

[probonopd]

Yes, I'd say that's what we do as soon as we use more than just one runtime.

[probonopd]

Might also help @develar https://github.com/probonopd/AppImageKit/issues/322#issuecomment-269238953

[lurch]

FWIW, :+1: to having AppImageKit support ARM too (even if it's initially limited to just Raspberry Pi).

[probonopd]

@lurch have you tried building it on ARM? Possibly it is already doable now, I just haven't tried so far.

[jviotti]

I'm happy to try tomorrow and report back my results.

[jviotti]

Hi @probonopd ,

I've tried the master branch (since we're not using appimagetool in Etcher yet) on an armv7l Raspberry Pi 3 running Raspbian Jessie, and it works out of the box (so I assume the new one does as well). I was able to package Etcher as an AppImage and run it successfully on the Pi.

Since our build scripts fetch AppImageAssistant and AppRun from https://github.com/probonopd/AppImageKit/releases/tag/6, maybe we can upload the ARM binaries I produced today there, at least until we move to appimagetool. What do you think?

I have every file listed there for armv7l however I'm not sure how you generate the .zsync files.

Here is the source tree after compilation, in case it helps: https://drive.google.com/file/d/0B7tkbonGU-RyNWhwOWRnREpuR3M/view?usp=sharing

[probonopd]

@jviotti is there a way to do the builds on Travis CI in a qemu armv7l chroot/container, e.g., using qemu?

[jviotti]

@probonopd Yes there is. We are planning to setup this for Etcher anyway (to create Raspbian builds), so I'm happy to back port my changes to this project as well once I'm done with it!

[probonopd]

@jviotti woud you be interested in/able to build the master/appimagetool branch for ARM?

[jviotti]

@probonopd Definitely. Resin.io has tons of ARM Docker base images that run on x86_64 hosts, so it should be a matter of making use of one of those (https://hub.docker.com/u/resin/). I noticed before that you're building appimagetool on CentOS. I don't think we have CentOS Docker containers, so would Ubuntu or Fedora be a good choice?

[TheAssassin]

@jviotti use the oldest available Linux distro you can find. Which one doesn't matter. CentOS is only used because it still works well, and ships with a really old glibc version. The glibc version should just be as old as possible. Debian 7 should still be available for Raspberry Pis, which is "old enough", so I'd recommend building on that. Fedora and/or Ubuntu likely ship with newer glibc versions.

[jviotti]

@probonopd, Sorry for the huge delay on this. Turns out we use slightly modified Docker versions to be able to run ARM containers, and we're still figuring out how to run these on Travis CI ourselves. Looks like we have a way forward, so I'll send a PR as soon as we figure it out!

On Tue, May 16, 2017 at 12:38:19PM -0700, TheAssassin wrote:

@jviotti use the oldest available Linux distro you can find. Which one doesn't matter. CentOS is only used because it still works well, and ships with a really old glibc version. The glibc version should just be as old as possible. Debian 7 should still be available for Raspberry Pis, which is "old enough", so I'd recommend building on that. Fedora and/or Ubuntu likely ship with newer glibc versions.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/probonopd/AppImageKit/issues/304#issuecomment-301892488

-- Juan Cruz Viotti Software Engineer

[probonopd]

@jviotti not sure if using the Open Build Service might be an option, but the public instance at build.opensuse.org can be used for free by Open Source projects. It can churn out AppImages for Intel and ARM, 32 and 64 bit each.

https://git.io/obs-ai