Self-compressed binaries?

[illwieckz]

@slipher proposed we may ship bare binaries instead of shipping them within .zip files. What if we use a self-compressed format like UPX?

I made some tests:

11M updater2.7z
12M updater2-upx
14M updater2.zip
32M updater2
136M    updater2-nonstripped

[DolceTriade]

I had looked into this earlier. The rationale for not picking this was that it can register as a virus sometimes and 1 MiB doesn't really warrant that risk. .zip compresses pretty well and is supported by default everywhere without extra tools.

[slipher]

These scans don't look too bad - https://github.com/upx/upx/issues/437

[illwieckz]

One good thing about the zip is that we can give any name to the zip, like UnvanquishedUpdater_0.6.0+linux-amd64.zip or UnvanquishedUpdater_0.6.0+beos-m68k.zip that would both contain a single file named Unvanquished Updater whatever the architecture, system or version.

[slipher]

For the next updater release, I'm thinking we should distribute UPX binaries for Windows and Linux on the download page. As a backup in case those binaries don't work for someone, we can have the standard binaries in zip files on Github.

I plan to make a PR adding the zip files and UPX binaries to the Docker builds.

[Calinou]

UPX has a bad track record with false positives on antivirus programs. We used it for Godot back in 2015 and ended up dropping it a few months later. I would recommed avoiding it in 2021.

Web browsers tend to display scary warnings when downloading unsigned executable files (compared to ZIP archives).

Also, if your distribution is already compressed in some way, there's no real point in using UPX.

[slipher]

Something I should have asked - does distributing a bare executable for Linux even work? I don't know whether there's any way to get it to download with the executable bit set.

[illwieckz]

Executable bit is meant to be stored on the storage layer, so distributing bare executable on Linux requires the user to flip the executable bit himself. Zip can carry that bit though (we may add this to the release validator as not every zip supports it, it's an extension as far as I know).

It looks like the only supported platform where distributing bare binaries would work out of the box is Windows. MacOS requires zip because the app is a directory.

[slipher]

I guess Linux is just doomed to be inconvenient then.

Mac is all right, because it has a default behavior of automatically unzipping the zip to a non-temporary location.

[illwieckz]

Note that UPX-compressed binaries may have issue about finding their “running directory” (or their own executable file path).

I attempted to compress netradiant with UPX and because netradiant relies heavily on self-location to look-up for required plugins, netradiant crashed badly.

So, we may face issues with UPX binaries when “self-installing” an installer by copying itself from self-location to somewhere else.

[slipher]

UPX doesn't work on the Linux updater.

$ upx updater2
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2020
UPX 3.96        Markus Oberhumer, Laszlo Molnar & John Reiser   Jan 23rd 2020

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
upx: updater2: CantPackException: bad DT_GNU_HASH n_bucket=0x46  n_bitmask=0x4  len=0x1d8

Packed 0 files.

[slipher]

Let's abandon this idea since Windows is the only platform where it's both technically possible and has significant benefits, and due to the UPX antivirus issues.