Auto-updating mechanism / Rolling Release

[Acuerta]

Have you considered going "rolling release"?

I'm just wondering because whenever there was a new sauer release it felt like we lost a good 30% of the playerbase to the outdated version. That aside, it would also make the game more dynamic since you can feed content like new maps to the users on a more regular basis and you can correct bugs.

I'm sure there are technical implications, so maybe other people are more suited to continue this discussion.

[aschaeffer]

+1

Yes, we want to deliver content faster. First step is to create a package system "around" the content. Next steps will include to connect to remote content repositories. Maybe using mongodb's gridfs, maybe using XMPP, maybe both (we didn't make a decission yet).

Also a new release shouldn't depend on new content (maps) anymore. It may be possible that certain game releases may be compatible to older and newer versions of the game. Whether it's actually possible depends on which changes has been made. If for example only bugs were fixed or a feature was added to the edit mode there are no gameplay changes or structural changes which makes the release incompatible to the previous.

Maybe semantic versioning would make it clear, which releases are compatible:

major.minor.bugfix

• Major releases are incompatible to each previous major release (gameplay / structural changes)
• Minor releases got new features which are still compatible
• Bugfix releases only fixes bugs, no new features

[Fohlen]

I entirely agree with @aschaeffer

Semantic versioning like major.minor.bugfix is totally fine. In this case we also want to deliver the new builds to our user-base which can possibly be resolved by:

• "forcing" them to download the new release from website (e.g: show a "new version available" message)
• writing an own updater similar to e.g Teamspeak's updater
• releasing packages on major platforms (distro-repositories for Linux, Packagist for Mac, ...)
• using a (more-or-less) open Gaming Plattform's like Desura to keep (mainly) Windows users on track

[Croydon]

As a side note: Red Eclipse has a .bat/sh file for the update mechanism https://github.com/red-eclipse/base/tree/master/bin

Imo that looks really painful to maintain. Perhaps we can cooperate with the Red Eclipse Community to find a (better) common solution? :)

[a-teammate]

Better solution: https://github.com/inexor-game/code/wiki/Release-and-build-strategy

Idea: use npm to ship and update inexor

[a-teammate]

Besides the packaging we need to have an automatic way of versioning. So the first step will be to let all CIs set the version for the package. Now we have different possibilities to calculate the version (reminder: numbers are for machines, names are for people)

1. Increment the version on each push to the master branch
   • Pro: easiest, does not require complex calculations just on every closed PR update the version
   • Con: no support for semantic versioning (no info about the impact of the changes in the version number)
2. Calculate the semversion based on the PR description ("#bigbump", "#bump", "#fix" or sth similar)
   • Pro: semantic versioning
   • Con: needs a tool checking whether there has been a closed PR.
   • Pro: Less often a decision about the impact has to be made than 3.
3. Calculate the semversion based on the commits pushed to master (each commit has to have a category/quality assessment attached, see 2.)
   • Pro: Semantic versioning
   • Con: needs a tool (but calculates it on easier reachable info than 2)
   • Pro: The tool is already written
   • Con: every commit needs to have a categorized commit message (?) -> automatic verification needed

I don't know whether we should first do 1) and in a second step switch to 3) or do 3) directly, since 1) requires to read the amount of commits made to the master branch and similarily 3) requires just a simple script to run and return the version.

[a-teammate]

This is a copy of the wiki entry of Fohlen:

Release and build strategy

The following article addresses main concerns and how to solve them during alpha release phase (current). The pad for this article can be found here.

Short release system

Proposed system will bring releases more often and fast:

• every approved PR will bring a new release
• depending on it's severity, the release is either a patch, minor or major according to semver
• major releases will be called out for new Inexor series (names)

The Inexor installer

The following proposal should make it easy for everyone to obtain Inexor:

• the core client and server will be npm modules
  • shipped via npm/update
  • built for Ubuntu 16.04 and Windows only for now
  • x64 is enough for now we think
  • building and publishing could/should be done via Travis and Appveyor
  • how is it possible to ship build stuff from conan?
• flex will be a npm module
  • flex has a standalone user interface
  • is opened by default in a Browser when core is not installed
  • contains: Install, Manage Plugins, Manage Instances

Release plan, why

We've been seeing a much decreased rate of commitment to Inexor, mainly because "the average user" cannot see/touch/use Inexor at all. That should be changed.

What's to be done:

• for team organisation: priorise tickets
• build a fast release cycle/system
• add the Flex UI
• go on with usual tickets (Refactoring, CEF bugs)
• add the information on the website!

Graphical illustration

Maintain new releases

Since not everything will be done automagically right in the beginning, we gonna start out maintaining the first few releases manually.

Bundle name

The default bundle name for now is generated by CPack, whereas the VERSION will be appended with an -alpha suffix

Building with CMake

Invoke conan with scope create_package=1 and then use make package install the usual make install. This will make a new inexor-..tgz archive.

[Croydon]

(repost https://github.com/inexor-game/code/issues/400#issuecomment-304734941)

1. Updating

So Inexor Flex is planned to update Inexor core. Flex can maybe get updated via a .sh/.bat file. How are we updating Node.js? NOT updating Node.js or expecting the users to do it manually is unacceptable.

2. Packaging problem

This doesn't fix the packaging problem for Linux or Windows (at all?). Quote Wiki: "built for Ubuntu 16.04 and Windows only for now" So it's highly limited on which Linux Distributations you can run it and we might get non producable "but it's working on my machine" bugs because of different Ubuntu versions AND we exclude all other Linux users which aren't Ubuntu users. Doesn't look like a solution. We are back were we started on Linux: Building for each and every distro binaries (not do-able) or exclude people (not acceptable). And if we are starting to build specific binaries, we could host distro specific files/repositories in the first place. Remember where we started this entire discussion?

Snapcraft as proposed by Fohlen previously or Flatpak (as favored personally by me #395) or another container-alike packaging format seems to be the only sophisticated, long-lasting solution on Linux. Everything else might improve a few things here and there, but is not solving it at the root of the problem, plus creating likely new problems on every step of the way. Snapcraft/Flatpak also having their updating services, solving the update problem as well. Also solving it in a widely adopted and tested way.

I'm sure we would find a good updating solution on Windows as well if we don't rush it. We are mostly a 4-core-team-member show and I feel sometimes like we want to do too much in parallel. Concentrating on doing it from the ground up might be better. Get our build system to the point that most people can easily build it, get out system to the point where everyone can run it easily, then get it packaged and auto updating for everyone. What we are doing right now in parallel is getting it build for some, have it complicated and error prone to run for everyone, and working on it to get it packaged for some.

3. Pre-installed End-user dependency

On Windows we seem to settle even more on Node.js/npm as an end-user dependency, right? (#364) How else should this setup work without a pre-installed Node.js/npm? And if we are starting again to ship Node.js/npm as we previously did with our platform repository (https://github.com/inexor-game-obsolete/platform) how are we supposed to update these via npm? (Back at problem 1.)

[aschaeffer]

(repost #400 (comment))

I don't believe we can deliver a rolling release system with 0.9.0. A lot of tasks have to be done until we get to this point. So this topic shouldn't the main focus of 0.9.0.

Until now the main focus were Flex, CEF and RPC/Tree, which are big goals already and all of them still needs work to be completed. I would suggest to focus on these and bring them into a stable state first before targeting another huge goal.

[a-teammate]

On 1.

flex will also be updated via npm.

true, one needs to update node.js as well. a bat on windows and a script on linux are enough for this (Since we are targeting Windows 7+, nodejs is retrievable via powershell and/or chocolate)

On 2.

Well the wiki is wrong.

On windows we don't have any problem. Actually on windows there gets a Inexor-version-alpha.exe and a Inexor-version-alpha.7z created every time Appveyor finishes. NPM would just replace the need to download and unpack it manually and do it automatically.

On linux it might seem more complicated at first sight but it isn't: With conan we have every dependency build and packed. So it can work as self-contained package like on windows. Or are there any edge-cases im not aware of? It's the same as linking everything statically. Is linking everything statically a portable linux binary?

The only problematic thing is CEF. since we do not have its dependencies build with conan, so we can't ship them in a self-contained package easily. The clean way is to port CEF to conan. since thats a lot of work CEF is limiting us to the OSes its build for. As we fix CEF, we make Inexor truely portable.

On 3.

we could create an installer which download node.js and excutes our download procedure somewhen in the beta stage.

[a-teammate]

(repost of #400 comment) @aschaeffer

I don't think going rolling release is actually that far fetched. Yes, sure we had different plans for 0.9.0 but CEF, Flex and RPC/Tree are all more high-level features than the rolling release system is. Since we're working bottom-up it would make sense imo to tackle this first.

So the ordering is:

1. make the build system more stable
2. make testing / distribution easy (rolling release)
3. add features and systems (CEF, flex, ..)

[Fohlen]

On @Croydon's comment

1. Initially, the users must update Node.js manually (in alpha stage). Later on we can use recipes (such as linux packages or windows installers) to automate all the hassles of the batch scripts and updating Node.js. One step at a time is probably the best way to go.
2. From my expierence, these systems do not cover platforms as large as Inexor. It is the best concept, yes, but it is simply overkill. That's why none of us, in all these years, was able to produce a satisfying solution with them. (maybe in the future snapcraft could work). Building only for Ubuntu 16.04 in alpha stage will mean that we don't have to test so many linux distributions. Again, this is one step at a time. Who says we wouldn't want support on more platforms? Starting with the system that has the biggest market share however is not a bad idea.
3. Of course, Node.js will be an end user dependency. What's so bad about that? If you install other games on Windows, they will prompt you to install the whole Visual Studio SDK. Compared to that, an 11 MB big scripting environment, with which you can modify the game to it's core, seems quiet a good deal.

On @aschaeffer I do think that rolling release should be the absolute focus of 0.9 release. This will finally make it possible to test Inexor (including ourselves), and will most likely get new/old(inactive) users back into Inexor.

We're so close to getting all the fancy technologies working, but we still ain't able to get Inexor running standalone (without miyards of dependencies).

Once rolling-release is working, we can improve Inexor in a very fast speed, with the difference that users can actually test our game and give feedback (mostly sums up what @a-teammate said)

We've been discussing these issues for years now, and is the reason that Inexor commitment is decreasing (because there is "none" result that you could see).

It's also been the main feedback that I have been receiving by a lot of people, that it is so hard to even test Inexor, which is why most of them give up with Inexor.

This is why I think focusing on a stable build and distribution system should be our top priority.

[a-teammate]

The InexorCore packages for each individual platform get published as follows:

this updates the npm modules @inexor-game/core_windows32, @inexor-game/core_linux32, @inexor-game/core_windows64 and @inexor-game/core_linux64 and pushes its corresponding .zip files to GitHub/releases

Details

• The PR quality checks are enforced by GitHub: passing builds on Travis and Appveyor (but we could add other ones as well).
• Travis tags a new version just by taking the last tag it can find and increment its minor version.
• Travis and AppVeyor create packages using CMake (or more precisely its CPack part)
  • If you cmake with -DCREATE_PACKAGE=1 you get the same result.

We need Quality checks for InexoFlex as well! Currently InexorFlex is part of the inexor-game/core_.. package (well, the linked submodule of the code repo is). Any ideas how to detach the InexorFlex general part from the updater? Or is there another solution?

---

Off-Topic:

Btw I think we should rename https://github.com/inexor-game/code to https://github.com/inexor-game/core now, shouldnt we?

[aschaeffer]

@a-teammate [Off-Topic]: Yes, we should. Please create a new issue for this!

[Fohlen]

This is definitely done, see #452

[a-teammate]

Now we need a wiki page for that :)

[Fohlen]

That's true, maybe we should reopen this until someones transfers this into a wiki page?

[Croydon]

I opened a new issue for writing a documentation: #493