Packaging binaries within containers useful?

[lm8]

Was wondering if there was a plan for how to add binary files (as mentioned in the blog post https://trilarion.blogspot.com/2019/11/can-wait-to-finish-next-steps-with-osgl.html ). I've been wanting to put together a collection of FLOSS educational games and applications for a long time. However, still have not come up with a good way to share the programs with users. If the programs are built for Linux, then there are all kinds of issues with library dependencies on the system that may not work with the program. For Windows, it's easier to share applications, but you still need all the libraries. For GPL programs, you need the source to all the dependencies possibly even some of the compiler source code (for the compiler runtime libraries) and maybe some of the build system source code (so people can replicate the building process). The suckless.org site recommends building programs statically which gets around many of the issues with mismatching of library dependencies. A lot of methods to produce portable apps on Linux use some sort of image and include a minimal operating system (or with solutions like Docker a minimal operating system minus the kernel) and all needed library dependencies so they avoid dependency collision issues (without requiring static builds). Was thinking a minimal Linux system (possibly based on AntiX or LFS) with the desired FLOSS software added might be a good way to distribute things. Then, anyone could run the image in a virtual machine like VirtualBox or Qemu or it could be set up as a Docker image. The image would work on any operating system that supports running virtual machines. The idea might extend to Windows builds as well. Might be useful to put together an image with the Windows executables that will run using Reactos or possibly even a version of Linux with Wine installed. Users could run the programs using the image or copy them from the image to their Windows installation if they have a compatible version of Windows. It would be more work to put together a collection of executables in an image that can be run, but it would avoid library dependency issues on many systems and make it easier to comply with GPL licenses. If desired the compiler and build system could be included on the disk, so that anyone who wanted to duplicate building the programs on the disk from source could do so (which is one of the goals of the GPL license). Those are some of the distribution issues and solutions I've been investigating. Will be very curious to see what this project considers for binary distribution solutions.

[Trilarion]

You could have a look at how the more famous open source games do it. Many games in this list offer binaries. Typically they used to include all third party libraries for Windows and Mac, while delivering a source package for compiling the game/app natively on your system before start on Linux. Nowadays they also deliver some more integrated Linux packages like what https://flathub.org/home offers. It also depends on the platform, a web app needs a different packaging than a desktop app or a mobile app.

Delivering the game in a whole container like Docker or VMWare / Wine is afaik not typical right now. I'm also not convinced that it's necessary right now, but if people want that and if there is a decisive advantage visible, why not.

First step should be to make the build easy and reproducible. Using a build system that is cross-platform like CMake for C,C++ projects, Gradle for Java projects and so on might be a good idea. From there on, it's what is most convenient for the programmer and the user.

In the beginning I would just reproduce what the other successful open source projects do. But if you want to experiment with other stuff, feel free to do it. It's open source.

[lm8]

If the program is GPL licensed, one would need to make available all the source code for each of the libraries as well which can get rather complicated. Takes about 30 libraries just to build a C/C++ program like Tuxmath. If you have several applications that use a lot of libraries, you can compile statically or you will need to include each of those libraries with each executable. If different versions of the same library are used on a system, one needs to be careful not to overwrite the other versions and to set the path properly to find the right libraries for a particular application. That was the main reason I thought a collection might be easier to maintain. There's just one copy of all the same libraries in one location. The path could be initialized once and point to a common location. Compiling everything statically avoids the library dependency issue, but has other trade offs. Also, many program's build systems are not designed to encourage static compilation.

[Trilarion]

Usually the libraries are more permissive then GPL and a link to the library homepage plus license information of the library is enough, I think.

I understand what you mean. A collection of games in a standardized container could be easier to maintain, but there are other risks like for example maintaining and distributing games nobody wants to play.

Already with the "every-app-is-packaged-by-itself" model, which is the traditional model, you have synergy effects of specializing in building them. The building process between projects of similar dependencies is probably not so different. Transfering knowledge from one project to the next, sharing buld scripts etc. would already be an advantage.

For example, one you figure out how to properly include SDL, so that dynamic linking works on Windows, Linux and macOS, in one project, it should be the kind of the same in other projects as well.

My own experience is not very large (I have some experience with CMake and Gradle and produced games/apps on Windows & Linux). With Python I used cx_freeze or upload them to PyPI. Java now has this thing where you package only the needed parts of Java and Web apps (JavaScript based) basically only need a play page.

What is the best solution? I guess it's a matter of trying out some of the possible ways and then learn from it.

[lm8]

Sounds like C/C++ programs (which is what I typically prefer to concentrate on) may have different distributions issues that Java or Python applications.

Personally, I prefer using CDetect and the standard GNU make for build tools. Not a fan of cmake. It tries to do everything and typically can't find any of the libraries on my system. It's also a real nuisance to build cmake from source. GNU autotools works fine with MinGW, but it's overkill for most programs. If you're building a simple C/C++ program, besides the compiler, you also need Perl, M4, bash and the gnu autotools packages (autoconf, automake, libtool, GNU make) just to get something to build. I've found GNU make alone works fine for building most programs and if you need more sophisticated functionality to figure out what works on a system or not like configure or cmake provide, CDetect can handle that. It works for cross-compiling too. I also have a series of cross-platform build scripts I use (similar to slackbuild scripts) to automate the build process for the various programs I build from source. Otherwise, it could take too long to rebuild everything by hand when there are multiple software updates.

My understanding (from all the time I've spent reading about licenses) is that if the game itself is licensed GPL, you need to supply the source to anything that links with it and is not a part of the operating system. That would include source for libraries that are linked to it like SDL, zlib, libpng, libvorbis, etc., but could also possibly include source for the compiler libraries on systems like Windows where MinGW is not part of the operating system. LGPL is more lenient and you need to supply source for anything with LGPL license but not anything it dynamically links to. If you just link to the library homepage and the library is no longer available at some point, you can't replicate building the GPL licensed program. The GPL license is designed to try to ensure that a user can rebuild the program from source for himself/herself if desired. I've even read about one license dispute that occurred when a build tool used to build a GPL program had a license that was not compatible with the GPL. You might be able to contact the FSF for further clarification on what source is absolutely required when distributing a GPL program, but I haven't found them to be any good at answering e-mails.

I can think of one other suggestion for a possible distribution channel at least for some SDL based programs. Emscripten could be used to make the programs available over the web (in HTML, CSS, JavaScript format). They could be hosted right on GitHub and I believe that would also take care of the source issues since the converted JavaScript source would be accessible (or the web app would not run).

[Trilarion]

CMake is the most popular build tool for C/C++ game projects in this database. Doesn't mean it has to be the only alternative (relying on a single tool to do so much work might actually be risky). Other alternatives beyond make or custom written scripts seem to be premake and meson. CDetect I didn't know yet.

Ideally we would also introduce CI to the projects, i.e. automated building (and testing) after each commit.

With the GPL license requirements that you describe, I have to do more research on it. For now I just hope that GPL software is not a nightmare in building and distributing.

Emscripten sounds very promising. We can surely try that. I hadn't really heard much of it before. It sounds cool if it works without much additional work (usually you have to adapt at least the UX when going to a different platform). Thanks for pointing that out. Feel free to experiment with Emscripten and please report back and share your experience.

[Trilarion]

I started a new issue about the GPL discussion (see #231) and another about applicability of Emscripten on these games (see #230) in order to allow a structured discussion.

[Trilarion]

For Linux Flatpack (https://flatpak.org/) is definitely an option.