search

MovingBlocks / Terasology

Terasology - open source voxel world
http://terasology.org
Apache License 2.0
3.67k stars 1.34k forks source link

Improving packaging of the whole game #2865

Open qwc opened 7 years ago

qwc commented 7 years ago

Due to the recent talk at slack about packaging I thought about writing down my thoughts on packaging of terasology. Forum discussion thread

Terasology Variants

First of all I would see 3 package variants of terasology

  1. The Launcher - the universal desktop approach, easiest for beginners too
  2. Latest Stable (Omega release)
  3. Latest Unstable (Omega release)

2/3 Would be best for setting up servers without docker. On Linux if the user wants to get updates through the distribution and not with some manual clicks inside the launcher. (May also be good for ppl with slow internet connections to get the latest version with the distribution update instead of when they just want to play and realize they have to download first to play)

Package signing

I think it would be good to also implement package signing so that downloaded packages can be verified.

Things to discuss here are

Linux distributions to target

Did I forget a distribution type with reasonable user base? O.o

@qwc is working on these.

Linux distribution independent packaging

@qwc is working on these also

Windows packaging

@gianluca-nitti is working on this.

Apple MacOS

Well, here I have quite a bit of a lack of knowledge. I know there are at least four things to consider:

Expert knowledge needed! @skaldarnar put homebrew on his list.

docker

Finally the docker solution for servers and hosting platforms, provided by my docker-terasology project

Maybe with predefined configurations and more customization in the future, needs to be discussed.

Important: automation

Everything mentioned must run through an automated job on a jenkins instance of the project (I'm so brash to include my own instance here).

Final words...

So far my thoughts to how the future variants could look and which package formats to target.

Human resources thoughts and claims for this tasks: So, come what may, but I'll at minimum claim the docker kill, that was my first contribution for terasology and I'll keep that baby. ;) I am familiar with all mentioned linux distributions and already packaged software for all of these (although it may be some time since then...) so I can do that basically, or may mentor ppl who want to do them. snap and flatpak are new to me, but I know basically what those projects want to achieve so it's not that hard to understand those formats.

On Windows - I'm almost gone, but already wrote some installers (WiX, and an other I cant remember...). On Mac - I'm completly gone.

An own repository for the packaging scripts may be needed, at least for all those linux distros I would wish for one. One for Win and Mac and we're done.

gianluca-nitti commented 7 years ago

I'm a Windows and Chocolatey user and if it helps I can tell that Chocolatey packages are almost always very simple scripts which download an installer (exe or msi) from an external URL (usually from the software's official site) and execute it in "silent mode" (without displaying windows / asking for confirmation), so I'd suggest to build an installer for the launcher and publish it somewhere, perhaps on the launcher repository releases so that it can be downloaded either from a website link or from Chocolatey (building a package is quite simple). It is also possible to create a package which actually contains the software files without downloading anything (self-contained) however I personally don't see big advantages in doing this if we consider that most Windows end-users don't use Chocolatey and thus a "normal" installer is probably important anyway. To build the installer I'd suggest NSIS , you configure the installer with a simple scripting language and build it with makensis, which also runs on Linux (popular distributions have it in their repos) so should be easy to integrate in the Jenkins build. Generating the Chocolatey package on Linux should be feasible too (choco is based on the .NET framework and Mono should be able to run it) but I've not tried personally.

As for Linux I'd like to mention fpm, not sure how it compares to the other solutions you suggest but I used it to generate deb and rpm packages and I found it a useful tool.

qwc commented 7 years ago

@gianluca-nitti Thanks for the clarification on Windows and Chocolatey. And a second thanks for pointing out fpm, I can remember reading about it, but forgot it until now.

Regarding windows, I would gladly give away the task to a Windows crack, instead of setting all up from scratch just to build an installer. :)

pollend commented 7 years ago

Have you looked at Appimages. An Appimage is a single file executable for linux. It basically mounts the file like a drive and executes the application. I think it would work fairly well for the launcher assuming that most people have java installed. It's possible to throw an error if java is not installed.

http://appimage.org/

skaldarnar commented 7 years ago

I can put homebrew somewhere on my todo list in the hope I'll find time to look into it. Brew is the only source of applications for me running my Mac, so I cannot say anything about other packaging variants (in addition, I'm new to Mac and have been using it for a few month now). In other notes: we had the idea of bundling a JRE with the game (and/or the launcher) to make sure everything is available to run the game properly. Can we make use of package managers and package repositories to model dependencies without bundling them (at least on Linux)?

gianluca-nitti commented 7 years ago

Can we make use of package managers and package repositories to model dependencies without bundling them (at least on Linux)?

+1 for this, IMHO bundling increases the size of the package without adding much real benefits. It can be done on Windows too, Chocolatey supports dependency management and we could make the Terasology package dependent on JRE 8+. The installer I'm working on can detect the JRE, so users which just download the installer's exe without using Chocolatey would be asked to download it from the official Java site before proceeding (screenshots here).

qwc commented 7 years ago

Participation, YAY! :)

Can we make use of package managers and package repositories to model dependencies without bundling them (at least on Linux)?

That is exactly, what I were aiming for! I've currently the linux packaging for all major distributions on my agenda. And of course improvement of the docker image.

@pollend Thanks for adding appimage to the list! Will try to include that aswell. @skaldarnar Yay! A Mac user, thanks for taking over at the Mac part. ;D

My goal would be to NEVER bundle the JRE, but adding always a dependency. On Linux that's easy. On Windows it has to be checked, thanks to @gianluca-nitti we'll have a nice installer doing this. On Mac, it's still unix so I can think of possibilities but in the end regarding the Apple way, I have no idea.

gianluca-nitti commented 7 years ago

I have another consideration, I'm not sure if I should post it here, on the forum thread or on my installer's PR. Writing here since I think this can be relevant for every platform.

As @qwc says in the opening, for desktop clients the launcher is what should be packaged for desktop clients (opposed to headless servers) since it allows to switch between versions easily. I just found out, however, that the launcher has an auto-update system which is used to self-update the launcher files. However, in most setups these would probably be put in system-wide locations (e.g. I imagine something like /opt in Linux, and my Windows installer currently uses %programfiles% which expands to C:\Program Files; no idea about macOS) which are usually read-only for a normal user; thus, the self-update wouldn't work. In my opinion the possible solutions are:

The second one feels weird in my opinion, updating a package is usually done by package managers or installers.

qwc commented 7 years ago

@gianluca-nitti Good point. I think I would go for 1. because, as you already wrote, the package update is managed through apt/pacman/emerge/yum/yast/whatever. And for the non packagemanager variants just enabling the autoupdate method.

skaldarnar commented 7 years ago

Disable the self-update in the launcher when it is built for packaging [...]

@gianluca-nitti That sounds sensible, although I think it is more feasible for package managers than for standard Windows installations. You usually end up with users having pretty old versions of the launcher because they think they don't need a newer release right now, and then they run into problems. We definitely should point them to the updated version on startup, if present.

@qwc The launcher has some "special" dependencies to take into account. The launcher does not only require a specific Java version, but also the JavaFX runtime lib. It is bundled with Oracle JRE, but comes separate for OpenJDK (see https://www.archlinux.org/packages/extra/i686/java-openjfx/).

qwc commented 7 years ago

My ongoing work can be found at https://github.com/qwc/terasology-packaging These scripts need docker images I am currently providing manually, I'll include the Dockerfiles in the repo when they are stable. The images bring everything to package the game, multiple images are needed because it looks like I can't do everything with *buntu due to the background checking of rpm. That's also a reason the images are not finalized. The images will be able to be used as a jenkins slave (I've added a headless jre inside) and I am already providing one of these docker images as a jenkins slave to my instance.

qwc commented 7 years ago

Drawback on fedora(rpms): openjfx is currently not included into the standard repo, so I'll just package the game for this distro, leaving the launcher aside until this is solved somehow.

Looks like I have to put up a fedora instance for testing some time how this could be solved through another approach.

qwc commented 7 years ago

With the current progress (see forum thread) it's already possible to test the debian and rpm packages. Download here: https://jenkins.mmo.to/view/Packaging/

So everyone using a debian variant (Ubuntu, Mint, etc) or Fedora may now test the packages, please report any issues to https://github.com/qwc/terasology-packaging thanks!

qwc commented 7 years ago

Simple debian repository is up: http://terasology.mmo.to/deb/