Package building on Linux

Something that would be very handy would be a means to locally build a Linux package (i.e. a .deb that can be built/installed locally, not THE Linux package) that would make it easier to install/test/remove a given build. This may just be a documentation issue as the existing build process has a package build product that appears to be the input to the package building process but I haven't seen any docs on how to get to a .deb...

so @probonopd I just tried your Appimage, and I must say I'm impressed with this. The size is similar to the one of our windows installers, it also has an option to integrate with your environment and it works like a charm. Also everything works with a click and it's not intrusive to the system. I love it, and I would really be interested if we could provide this to the users, and if we could get an automated Travis task to do this.

Wrote an AppImage recipe that converts the deb from the PPA into an AppImage: https://github.com/probonopd/AppImages/blob/master/recipes/meta/LMMS.yml

so @probonopd I just tried your Appimage, and I must say I'm impressed with this.

If it doesn't contain the majority of dependent libraries then it won't work cross-distros.

Providing an AppImage would have, among others, these advantages:

Works for most Linux distributions (including Ubuntu, Fedora, openSUSE, CentOS, elementaryOS, Linux Mint, and others)

makeself already does this.

One app = one file = super simple for users: just download one AppImage file, make it executable, and run

makeself already does this.

No unpacking or installation necessary

makeself already does this.

No root needed

makeself has this ablity, and already discussed here.

No system libraries changed

makeself can do this.

Works out of the box, no installation of runtimes needed

makeself can do this.

Optional desktop integration with appimaged

makeself can do this, albeit manually

Optional binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate

makeself cannot do this.

Can optionally GPG2-sign your AppImages (inside the file)

I've never signed a .run before, so this is something I don't know about.

Works on Live ISOs

makeself can do this.

Can use the same AppImages when dual-booting multiple distributions

rare edge-case but makeself can do this too.

[you] didn't even bother reading the thread and the thoughtful, thorough comments other people wrote here.

Please don't ad hominem. Makeself uses tools available on more distros for a longer period of time so it truly is a lower common denominator regardless of which is preferred. We welcome the start of this, but the work is inevitably redundant with the .run task. The hard part is getting the dependencies sorted, which requires a bunch of recursive copying and re-linking. If linuxdeployqt (comment) makes this simpler, let's get a prototype PR going.

We also need to know which OSs ship with AppImage support out of the box and which don't. We need to know our deployment scope.

makes it clear you have no idea what you're talking about

Having installed a .run on old, new and obscure OSs and used .run files as an end-user by major companies like nVidia , VMWare, JetBrains we should focus on the technical differences. We really don't tolerate mudslinging here, so if we can keep the conversation civil and focus on the technology that will yield the fastest progression. If something I've said seems counterproductive to this argument, please focus on that so that we can strive for better communication.

At the end of the day, I don't believe these tasks are separate. Both system move shared objects and binaries into a directory so an application can be run. What container we wrap the files in is a mostly superficial decision because 1. To validate a "good" signature versus "bad" requires a chain of trust. This is the same on all OSs, really. 2. The diff or delta feature is only useful for auto-updates which I'm not sure we'll be providing out of the gate. So I stand by my original statements. I hope you take them purely as technological observations.

If your best argument is "makeself does everything that AppImage does" then why are you so invested against AppImage? You just proved they're both on par - except me and others here seem to think AppImage is a better solution. If you have tried to express any reason why makeself is clearly superior then it didn't come across for me and I, politely and nonadhominenly ask that you please try once more.

obscure OSs

The fact that run files are supported on legacy systems is moot. LMMS doesn't need or should care about old systems. Linux isn't like Windows or Mac, you can always upgrade your system, with plenty of up-to-date options for legacy hardware. Also, LMMS isn't a commercial project, even if one or two people simply cannot update their systems no matter what, doesn't mean this should become a development priority - quite on the contrary: it's a time, effort and energy sink while 99,9% of the users are running modern Linux distros and will be able to use both run and AppImage regardless.

nVidia , VMWare, JetBrains

So what? The use case is vastly different. They're installing software, which I agree .run files are perfect for and AppImages aren't. That's not what the AppImage discussion is about. LMMS already has packages in several repositories and a PPA for system installation. Meanwhile, AppImage is becoming a fast standard for self-contained Linux package distribution, which seems to be what it's being discussed for.

Trying to advance the discussion, though: honestly, does anyone here see a problem offering both an AppImage and a run file? There seem to be several people here in favor of offering an AppImage and one in favor of a run file. The OSS ethos values action, not talk. Let each party build their own build scripts and offer both packages for download on the website. This thread seems to indicate an AppImage is already being produced, let whoever wants to create a .run create a build as well.

I'm not saying that LMMS should indeterminately and officially maintain two build scripts - just that as long as someone is willing to contribute them in a 100% ready state, there is no downside to offering both files for download. With time, it will become clear which one has the most technical merits or shortcomings and in the future it will be easy enough to drop one of them and maintain the other one if that's necessary.

Also everything works with a click and it's not intrusive to the system

I don't think that makeself can beat this. As far as I understand it extracts files into a temporary folder instead of mounting an image like AppImage does. This means that it's pretty easy to run into free-space issues, especially if the run file uses /tmp/ by default, which many distributions use a separate (and thus fixed-size) partition for - not to mention "live" distributions which would have to unpack those directly into memory instead of reading directly from whatever storage the AppImage is located at.

This is the wrong place to cast a vote for a package bundler. If there's limitations, point them out. Anything else is simply wasting valuable developers time. Please stop arguing and keep the content technical or this thread will be locked in favor of a PR.

Now to answer the actual content of these long-winded replies...

There seem to be several people here in favor of offering an AppImage and one in favor of a run file.

This isn't a democracy. We accept code that works and the admins that run the project try their best to weight the differences when a conflict arises. Opening a PR is a good way to prove a point. Then we can test the code instead of speculate about it.

Also everything works with a click and it's not intrusive to the system

I don't think that makeself can beat this

We'd all love that, but I remember something being said about setting the file executable. If that's true, it's not a single-click. If we can set executable flag prior to download (or buy a trusted certificate to unlock this) let's talk about that. It's relevant.

Although I do agree that double-clicking a file is easier than executing it through terminal, it's still not correcting the packing problems which will occur regardless. It took quite a while to fix the linking and bundling problems with Mac, so if we can focus on the viability of the package instead of the container format, we'll be closer to offering this as a download.

this means that it's pretty easy to run into free-space issues, especially if the run file uses /tmp/ by default

If /tmp fills with a < 100MB installer, the user has much bigger problems. It's a nice safeguard, but shouldn't be considered a show-stopper. There is an equal and opposite edge-case where the main volume lacks the space as well. Let's not bikeshed over /tmp.

"live" distributions which would have to unpack those directly into memory

This is true for all storage on a live partition. Not sure how this is different. Has someone tried our .run installer on a live CD and proven that it runs into such /tmp problems? I've installed .run files on ChromeOS which has a read-only /tmp and managed to get it to all work silently, seamlessly. The technology does work.

AppImage is becoming a fast standard for self-contained Linux package distribution, which seems to be what it's being discussed for.

I think this is a good selling point but we should also identify projects which have adopted it, especially if they're Qt based so that we can learn from their packaging system.

I'm not saying that LMMS should indeterminately and officially maintain two build scripts

Well, they aren't necessarily two separate scripts. If an AppImage tool can bundle better (e.g. linuxdeployqt), we can wrap the outputted directory in whatever container we choose (tarball, .run, or AppImage). They aren't mutually exclusive.

We're just waiting on someone getting a successful bundle. Wrapping the .deb into an AppImage isn't necessarily the solution to that, we'd want to make a pure AppImage and chain it into a CMake packaging target.

I remember something being said about setting the file executable

I installed an nvidia driver yesterday (they use .run files) and had to set the execution permission manually. makeself clearly doesn't have an answer to this. Unless you're packaging the file on a .tar.gz or similar format that supports +x then there is no way around this as far as I know - and of course, you shouldn't be distributing an AppImage or .run file inside a compressed archive regardless - and if you were to do it, doesn't matter which format the file inside is.

There is no easy way around this because it's a Linux security feature, making this point near-irrelevant. If it was very easy for people to just automatically set +x to any Linux download it would simply make the entire point of having a permission useless. If that is a big problem for people here, put the AppImage or .run file inside a .tar.gz, even though it'd seem very silly to me.

If /tmp fills with a < 100MB installer, the user has much bigger problems

Today. Who knows what the installer size will be 2 years from now? Nobody can predict the future. This becomes an even bigger issue on live distributions because no matter where the data is extracted at (/tmp, /home, etc), it wil reside in memory, which is almost always the rarest resource on any live session.

This is true for all storage on a live partition. Not sure how this is different.

If you're not sure then you missed my point entirely: people using Live distributions, very often, will pack their "mobile" apps on a removable device or into the live media itself (if they've built it themselves), avoiding having to redownload whatever applications are needed after every boot. This means that with an AppImage the files reside only on a USB drive or the live media itself (say, a DVD) while with a run file it would have to also be extracted into memory before use.

If you're thinking about the use case of a live CD user downloading the entire package from the website after every boot, then, in the case of a run image, the user needs to have it downloaded (100MB), extracted (another 100MB or more) and then running (another 100MB or so, I think). Your "small" 100MB package just became a 300MB+ memory hog in this case - which is definitely a respectable number for a modest computer trying to run a full desktop environemnt, plus any other apps needed.

Anyway, I'm not sure if Live sessions should be a concern at all for LMMS, I just brought it up initially because someone mentioned them here prior. It's easy to see though that AppImage makes this a complete non-issue, making it more suitable, at least in this case, for self-contained execution.

Wrapping the .deb into an AppImage isn't necessarily the solution to that, we'd want to make a pure AppImage and chain it into a CMake packaging target.

Explain to me why, when @probonopd already has a working AppImage, verified by yourself, using a very simple YAML recipe? You were advocating for keeping this discussion strictly towards the benefits of each packaging method. What technical advantages does building an AppImage from scractch have over a deb-based one?

What technical advantages does building an AppImage from scractch have over a deb-based one?

Two reasons...

We don't yet have a deb packaging script.
Because a deb package is a distro-specific package format. In short, debian packages generally don't bundle what they need, they fetch them. Thus, wrapping an LMMS deb file into yet another package format is a bad permanent design.

I installed an nvidia driver yesterday (they use .run files) and had to set the execution permission manually. makeself clearly doesn't have an answer to this.

No one made claim that it did. It was only a rebuttal to the glittering generality of this statement.

" just download one AppImage file, make it executable, and run

If you're not sure then you missed my point @probonopd already has a working AppImage, verified by yourself

Lets, please avoid ad hominem, it never ends well. And I didn't verify anything was working, BTW.

Who knows what the installer size will be 2 years from now?

Let's use arguments we can safely measure, but to entertain the question, the project maintainers have a pretty good idea. It forces us a bit off-topic to explain, but we at least have the best idea as we've seen the project grow for several years, so we are in the best place to make an educated guess. 0.4.5 was released in 2009 at 17MB. 1.2.0 will be released in 2017 at about 30MB. Size varies about 5-10MB between platforms, so in 2 years the file size will be a comparable gain. About 5MB of that gain can be directly attributed to switching from Qt4 to Qt5, btw.

Your "small" 100MB package just became a 300MB+ memory hog

Taking 100MB hypothetical out of context and arguing with it is misleading. 40MB * 3 = 120MB which is much more realistic and practical. Second, unless using the --keep option, makeself cleans this data up after itself, so it's really 40MB * 2 = 80MB. Third, users should cleanup downloads, that's their problem not ours. So from a disk utilization perspective, the net amount is going to be about the same with delta minus the storage difference between compressed versus uncompressed. I would expect the compressed archive to be a smaller size but only at the cost of CPU cycles. I don't know enough about how FUSE AppImage performance compares against a plain-old-file so this is best measured once we have an image created.

Live distributions, very often, will pack their "mobile" apps on a removable device or into the live media itself (if they've built it themselves), avoiding having to redownload whatever applications are needed after every boot. This means that with an AppImage the files reside only on a USB drive or the live media itself (say, a DVD) while with a run file it would have to also be extracted into memory before use.

Thanks for the background information. This is an interesting use-case. I am more familiar with users baking software directly onto the medium. I can see how the FUSE support makes this less burdensome, like plugging in a removable drive and running.

I'm not sure if Live sessions should be a concern at all for LMMS,

I'm not either, but the live CD argument does raise some good points and the marketing appeal of making the latest LMMS available on a live medium has its merits.

So what I'm reading is that AppImage is extremely similar to the way Apple does the DMGs, except:

AppImage saves some steps by skipping the intermediary mount, extract, unmount steps (whether this has performance implication is TBD)
Contrary to Apple, AppImage doesn't provide a way to click-and-run the executable without changing execute permission. At least not yet. If the distros agree on a way to digitally sign these to avoid the chmod +x step, this would put it on par with how Windows and Mac do it today and would be a major advantage AppImage would have over .run.

On a side note, having such a "Portable" installation does carry it's own caveats, mostly caused by working paths and relative paths, but we've done a pretty good job of coding for these caveats.

From a support perspective, writing the installer ourselves yields predictable install locations, which is a plus from the project's perspective. The vast user-base creates a lot of support chatter. Controlling the manner in which an application is installed and registered with the OS can offer some advantages.

Mac gets around this problem by just having a culture that expects everything dragged to /Applications. I'm not aware of such a place that exist for the Linux desktop, so we'd loose the predictability of deployments, which can also effect the upgrade path, if we ever employ one.

But this is all just chatter until it happens. If someone can prepare a PR for AppImage, we can start debugging and troubleshooting it together. Making it before the 1.2 release would really be beneficial to the community at large (regardless of the container format).

AppImage doesn't provide a way to click-and-run the executable without changing execute permission

Neither does .run files. In this sense, you haven't answered my question from a few comments ago asking you to point out if .run has any advantages over AppImages (legacy systems aside, as they're non-critical). You went to some length to prove they're both equivalent solutions and since then we've been doing nothing but discuss technical AppImage advantages. If you (or anyone) cannot provide an argument for .run files over AppImages it seems to me this discussion is then settled.

If someone can prepare a PR for AppImage

@probonopd can you create a pull request adding your recipe into something the project can use? You mentioned in a previous comment that you were willing to make it happen as a Travis artifact build, if there was any interest. I think it's safe to say now that there is a real interest here.

Making it before the 1.2 release would really be beneficial to the community at large (regardless of the container format)

Seems to me @probonopd is quite ready to deliver on that.

If you cannot provide an argument you haven't answered my question

Since the intent of most recent comments are around "being right" instead of "whats right" by the continual ad homium references, locking. Please pick technology camps elsewhere.

Superseded by #3558 (#3688). Some of the conversation in this thread is around packaging for a specific distribution and many of those issues are still valid, however those tasks are currently maintained by the OS packagers themselves. If this changes, we can open a new issue, or discuss on Discord.

LMMS / lmms

Package building on Linux #2556