Offer an AppImage for Linux

zilti commented 5 years ago

It would be nice to have an AppImage for the Linux users. It's a self-contained executable packed a bit similar to a dmg on macOS and would faciliate a lot.

HebaruSan commented 5 years ago

How would this differ from downloading and running ckan.exe?

politas commented 5 years ago

How would this differ from downloading and running ckan.exe?

You wouldn't need to have a compatible Mono installation.

HebaruSan commented 5 years ago

So we would bundle Mono into our package? Wouldn't that be hundreds of MB?

zilti commented 5 years ago

Hundreds? It's interesting how today's devs have completely lost all sense of space. mono-core takes up 22.5 MB. But 1. you don't necessarily have to include mono - see for example OpenRA. And 2. having an AppImage gets you as the user quality-of-life features like update checks, updates, and a menu entry in whatever launcher you use.

HebaruSan commented 5 years ago

I'm happy to admit when I'm wrong, but why be insulting about it? Remember that you're trying to convince someone to do something to help you here. :)

Regarding "see for example OpenRA", can you be more specific? Apparently it's a Red Alert clone, but what about it is relevant here? They don't seem to explain their Mono deployment strategy on their home page.

HebaruSan commented 5 years ago

As for the actual size, it's not clear that mono-core is enough to run CKAN. The size of the actual downloads from mono-project.com are obscured by the debian repo folder structure (where are the actual .debs??), but for comparison the Windows download is 129 MB.

zilti commented 5 years ago

What I meant to say is: OpenRA does not include Mono into their AppImage, so the user has to install it separately, is what I meant: "OpenRA is distributed as portable AppImages that run on most modern 64-bit Linux distributions. Mono 4.2 or later is required." Mono seems to be installed by default on most distributions anyway.

DasSkelett commented 5 years ago

But the problem with CKAN is, that we need a version of mono that is not installed by default on most distributions. And if th user has to install mono and download the AppImage, what's the difference to the install process right now, where he has to install mono and download the ckan.exe (or the .deb)? Seems like the user would just download a different file, to get the same, with the same effort.

Cyclic3 commented 5 years ago

DISCLAIMER: I have just hit #2664, so I may be biased against .deb archives at the moment.

The two things we would get from appimages is this:

Supporting all Linux distros with a single package, which would ease releases, and would make it easier for non-debian based users to install
Make it much easier to install. I have been using debian for half a decade now, and I still find it irksome to run sudo dpkg -i pkg.deb; apt-get install --fix-broken and then spend half an hour hunting for the missing dependencies. AppImages brings to Linux the one thing that Microsoft's Windows actually is better in: programs are easier to set up. Running an appimage just works, without any installation of any prerequisites (also allowing them to be installed without root)

Also, I would like to add that even though they are a lot bigger, most computers now operate in the terrabyte relm, and if someone changed it over tonight whilst I was asleep, I would not notice one package going from ~1M to ~100M.

zilti commented 5 years ago

Ok, so what mono components does ckan use exactly? I'm guessing more than just mono-core and mono-winforms.

zilti commented 5 years ago

I just went with what OpenSUSE considers "mono-core", i.e. the mono-core package and its mono dependencies. Here's a script that generates an appimage, expecting zsyncmake to be on your path:

#!/usr/bin/env bash
wget https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage
mv appimagetool*.AppImage appimagetool
chmod +x appimagetool

rm -rf ckan.AppDir
mkdir ckan.AppDir

wget https://github.com/KSP-CKAN/CKAN/releases/download/v1.25.4/ckan.exe

mkdir rpms
cd rpms
wget http://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/mono-core-5.16.0-1.1.x86_64.rpm
wget http://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/mono-extras-5.16.0-1.1.x86_64.rpm
wget http://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/mono-data-5.16.0-1.1.x86_64.rpm
wget http://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/mono-data-sqlite-5.16.0-1.1.x86_64.rpm
wget http://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/mono-mvc-5.16.0-1.1.x86_64.rpm
wget http://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/mono-wcf-5.16.0-1.1.x86_64.rpm
wget http://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/mono-web-5.16.0-1.1.x86_64.rpm
wget http://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/mono-winforms-5.16.0-1.1.x86_64.rpm
wget http://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/mono-winfxcore-5.16.0-1.1.x86_64.rpm

cd ../ckan.AppDir
rpm2cpio ../rpms/*.rpm | cpio -idmv

cp ../AppRun .
cp ../logo.png .
cp ../ckan.desktop .
cp ../ckan.exe usr/bin

cd ..
./appimagetool ckan.AppDir ckan.AppImage -u "gh-releases-zsync|KSP-CKAN|CKAN|latest|ckan.AppImage.zsync"

I've seen that you're using Travis; it should be possible to integrate that I guess. Both ckan.AppImage and ckan.AppImage.zsync then need to be put up on the GitHub release download page. AppImage.zsync is update info; if the user has AppImageUpdate, this will allow the AppImage to be updated.

Cyclic3 commented 5 years ago

@zilti When I get some time this evening, I will make a fork with a build script (unless someone else wants to have a go)

zilti commented 5 years ago

Alright. I've put everything into this gist: https://gist.github.com/zilti/b5937ebd68fcbb02f8de6b5b7c9aa79b

Cyclic3 commented 5 years ago

Mono's mkbundle appears to be similar to appimages in their functionality AFAIK. Does anyone have any reason why we shouldn't use them?

zilti commented 5 years ago

Because it's chronically broken.

Cyclic3 commented 5 years ago

I have tried using pkg2image, appimagetool and linuxdeployqt to no avail. I get, respectively, nothing, direct calling of mono, and segfaults. I do not believe I am able to complete this task.

@zilti AppImage itself recommends it, so that may present an issue.

zilti commented 5 years ago

Did you use my script I've put in the gist? I can execute my script on a machine without any installed mono and get a working AppImage.

Cyclic3 commented 5 years ago

@zilti I compiled the binary on a computer with mono, and copied it to my ubuntu/bionic laptop, which I have removed all traces of mono from. On execution, I hit this error.

System.TypeLoadException: Could not load type of field 'CKAN.RegistryManager:registry' (7) due to: Could not load file or assembly 'System.Transactions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies. assembly:System.Transactions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 type:<unknown type> member:(null) signature:<none>
[ERROR] FATAL UNHANDLED EXCEPTION: System.TypeLoadException: Could not load type of field 'CKAN.RegistryManager:registry' (7) due to: Could not load file or assembly 'System.Transactions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies. assembly:System.Transactions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 type:<unknown type> member:(null) signature:<none>

The mono mkbundle gives me System.BadImageFormatException: Method has no body

:(

DasSkelett commented 5 years ago

Well, I think you didn't include libmono-system-transactions4.0-cil (or whatever it's called on other distributions) in the AppImage. Look at our wiki to see what packages CKAN depends on: libcurl4-openssl-dev ca-certificates-mono libmono-accessibility4.0-cil libmono-addins-gui0.2-cil libmono-addins0.2-cil libmono-cairo4.0-cil libmono-corlib4.5-cil libmono-data-tds4.0-cil libmono-i18n-west4.0-cil libmono-i18n4.0-cil libmono-ldap4.0-cil libmono-microsoft-csharp4.0-cil libmono-posix4.0-cil libmono-security4.0-cil libmono-sharpzip4.84-cil libmono-sqlite4.0-cil libmono-system-componentmodel-dataannotations4.0-cil libmono-system-configuration4.0-cil libmono-system-core4.0-cil libmono-system-data4.0-cil libmono-system-design4.0-cil libmono-system-drawing4.0-cil libmono-system-enterpriseservices4.0-cil libmono-system-ldap4.0-cil libmono-system-net-http-webrequest4.0-cil libmono-system-net-http4.0-cil libmono-system-numerics4.0-cil libmono-system-runtime-serialization-formatters-soap4.0-cil libmono-system-runtime-serialization4.0-cil libmono-system-security4.0-cil libmono-system-servicemodel-internals0.0-cil libmono-system-transactions4.0-cil libmono-system-web-applicationservices4.0-cil libmono-system-web-services4.0-cil libmono-system-web4.0-cil libmono-system-windows-forms4.0-cil libmono-system-xml-linq4.0-cil libmono-system-xml4.0-cil libmono-system4.0-cil libmono-webbrowser4.0-cil libmono-zeroconf1.0-cil mono-4.0-gac mono-gac mono-runtime mono-runtime-common mono-runtime-sgen (on Ubuntu)

Cyclic3 commented 5 years ago

Why not just debboostrap a folder, install the relevant debs and package that?

zilti commented 5 years ago

Ok I'm using one of the precompiled mono runtimes now. I've updated the gist. I can confirm that what I posted before did not work. This now should.

zilti commented 5 years ago

So did this work out?

Cyclic3 commented 5 years ago

@zilti Sorry for the late response! I have been rather busy for a couple for days, and missed your response.

You have indeed fixed the error, but unfortuantely a new one has come up:

[ERROR] FATAL UNHANDLED EXCEPTION: System.TypeInitializationException: The type initializer for 'System.Windows.Forms.XplatUI' threw an exception. ---> System.TypeInitializationException: The type initializer for 'System.Drawing.GDIPlus' threw an exception. ---> System.DllNotFoundException: libgdiplus.so.0
  at (wrapper managed-to-native) System.Drawing.GDIPlus.GdiplusStartup(ulong&,System.Drawing.GdiplusStartupInput&,System.Drawing.GdiplusStartupOutput&)
  at System.Drawing.GDIPlus..cctor () [0x000b0] in <1eb3f7d2c1ad45e48eedc0b6e712e192>:0 
   --- End of inner exception stack trace ---
  at (wrapper managed-to-native) System.Object.__icall_wrapper_mono_generic_class_init(intptr)
  at System.Drawing.Graphics.FromHdcInternal (System.IntPtr hdc) [0x00000] in <1eb3f7d2c1ad45e48eedc0b6e712e192>:0 
  at System.Windows.Forms.XplatUIX11.SetDisplay (System.IntPtr display_handle) [0x00073] in <b5f2a8a643c2407cac092e48799ae08d>:0 
  at System.Windows.Forms.XplatUIX11..ctor () [0x00077] in <b5f2a8a643c2407cac092e48799ae08d>:0 
  at System.Windows.Forms.XplatUIX11.GetInstance () [0x00019] in <b5f2a8a643c2407cac092e48799ae08d>:0 
  at System.Windows.Forms.XplatUI..cctor () [0x00066] in <b5f2a8a643c2407cac092e48799ae08d>:0 
   --- End of inner exception stack trace ---
  at System.Windows.Forms.Application.EnableVisualStyles () [0x00006] in <b5f2a8a643c2407cac092e48799ae08d>:0 
  at CKAN.GUI.Main_ (System.String[] args, CKAN.KSPManager manager, System.Boolean showConsole) [0x0001b] in <2c2371fb3f2949b2a6dfae930758e6ee>:0 
  at CKAN.CmdLine.MainClass.Gui (CKAN.KSPManager manager, CKAN.CmdLine.GuiOptions options, System.String[] args) [0x00008] in <2c2371fb3f2949b2a6dfae930758e6ee>:0 
  at CKAN.CmdLine.MainClass.RunSimpleAction (CKAN.CmdLine.Options cmdline, CKAN.CmdLine.CommonOptions options, System.String[] args, CKAN.IUser user, CKAN.KSPManager manager) [0x002ab] in <2c2371fb3f2949b2a6dfae930758e6ee>:0 
  at CKAN.CmdLine.MainClass.Execute (CKAN.KSPManager manager, CKAN.CmdLine.CommonOptions opts, System.String[] args) [0x00187] in <2c2371fb3f2949b2a6dfae930758e6ee>:0 
  at CKAN.CmdLine.MainClass.Main (System.String[] args) [0x00091] in <2c2371fb3f2949b2a6dfae930758e6ee>:0

zilti commented 5 years ago

Ah, I see. I had a not minimal enough test environment. Updated, and tested in a barebones environment now.

Cyclic3 commented 5 years ago

@zilti I now get an error at build:

-- Deploying dependencies for existing files in AppDir -- 
Deploying dependencies for ELF file AppDir/usr/bin/mono 
  Skipping deployment of blacklisted library /lib/x86_64-linux-gnu/libm.so.6 
  Skipping deployment of blacklisted library /lib/x86_64-linux-gnu/librt.so.1 
  Skipping deployment of blacklisted library /lib/x86_64-linux-gnu/libdl.so.2 
  Skipping deployment of blacklisted library /lib/x86_64-linux-gnu/libpthread.so.0 
  Skipping deployment of blacklisted library /lib/x86_64-linux-gnu/libgcc_s.so.1 
  Skipping deployment of blacklisted library /lib/x86_64-linux-gnu/libc.so.6 
Deploying dependencies for ELF file AppDir/usr/lib/libmono-btls-shared.so 
Deploying dependencies for ELF file AppDir/usr/lib/libMonoPosixHelper.so 
  Skipping deployment of blacklisted library /lib/x86_64-linux-gnu/libz.so.1 
Deploying dependencies for ELF file AppDir/usr/lib/libMonoSupportW.so 

-- Deploying shared libraries -- 
No such file or directory: /usr/lib64/libgdiplus.so.0

Why don't we continue this at your gist comment section, so that this issue is not spammed with back and forth about the implementation?

BeatLink commented 4 years ago

Any update on this?

YellowApple commented 4 years ago

This would jive well with #2771, since .NET Core has built-in support for generating self-contained Linux (and macOS and Windows) deployments with a bundled .NET runtime, at which point it's relatively straightforward to package it all up as an AppImage.

zilti commented 4 years ago

It is up to them to finish this, I provided everything. Seems like they prefer maintaining three different packaging scripts over having an AppImage.

KSP-CKAN / CKAN

Offer an AppImage for Linux #2636