Kernel naming and update mechanism/procedure proposal

[ThanosApostolou]

Inspired by this: https://forum.manjaro.org/t/idea-kernel-metapackages-renaming/23037, I took a moment to think about kernel updating mechanism/procedure and took some time to do a more detailed proposal which can be discussed here.

Kernel Pipe

EXPLANATION:

A new kernel gets first into the higher position (linux-dev) and moves down the pipe or gets deprecated accordingly to whether it matches the criteria. This is about major kernel series updates (kernel maintenance point releases happen independently). More or less same thing is happening now, but currently user intervention is needed. Explicit kernel packages (e.g linux4.11) can still be built and included at repositories if some specific hardware requires only a specific kernel version (there shouldn't be any such cases normally), but they shouldn't be considered by the kernel pipe updating procedure and shouldn't be easily accessible to normal users but only for advanced ones.

ADVANTAGES:

1. Maintain and build only 4 kernels (and 2 rt).
2. Automatically update to next kernel series without user interference (msm notifier kernel notifications should be removed and keep only the kernel utility at msm settings for advanced users).
3. Before a kernel moves down the pipe, it will have to get into the higher position first. So, by this, we create a 2nd layer of testing, additionally to the unstable->testing->stable branching system, which assures quality and stability (see example and NOTE at the end).

EXAMPLE:

Time snapshot 1: We have upstream kernels: mainline=4.14-rc6, stable=4.13.9, lts=4.9.58, lts=4.4.94 Manjaro kernels should be: linux-dev=4.14-rc6 linux-current=4.13.9 linux-lts=4.9.58 linux-lts-perv=4.4.94

Time snapshot 2: Let's say that 4.14.0 gets released. Manjaro development kernel doesn't move down the pipe yet. So we have: linux-dev=4.14.0 linux-current=4.13.9 linux-lts=4.9.58 linux-lts-prev=4.4.94

Time snapshot 3: At this time let's say that we have upstream kernels: mainline=4.15-rc1, stable=4.14.3, stable=4.13.10(EOL), lts=4.9.60, lts=4.4.96 Manjaro kernels become: linux-dev=4.15-rc1 linux-current=4.14.3 linux-lts=4.9.60 linux-lts-prev=4.4.96

Time snapshot 4: If kernel 4.14 announced to be an lts kernel, then manjaro's linux-lts shouldn't be updated yet to linux-4.14.x series, but only when 4.15 is ready to get to linux-current. If 4.14 got EOL then linux-lts and linux-lts-prev don't change. In that moment we will have something like this: Manjaro kernels (if 4.14 is lts): linux-dev=4.16-rc1 linux-current=4.15.3 linux-lts=4.14.5 linux-lts-prev=4.9.62 Manjaro kernels (if 4.14 got EOL): linux-dev=4.16-rc1 linux-current=4.15.3 linux-lts=4.9.62 linux-lts-prev=4.4.98

NOTE: When linux-lts series change (4.14 series in our example), it was first tested enough when it was at linux-dev and then at linux-current, so it should be really stable. However, since all these changes happen first on unstable, then we got a 2nd layer of testing by users before it gets to stable branch (even point releases can cause problems sometimes).

[Strit]

But. Some hardware will only work with certain kernel versions, like old 3.18 etc. How would that be supported?

There's a reason why @philmmanjaro often asks which of the old kernels people use.

[philmmanjaro]

We don't go the Arch-Way here. Manjaro has a reason why we have those kernel series as they are right now. It is one feature of Manjaro to give the user the choice which kernel series he wants to install in parallel or not. So there will be no change in that. We can think about meta packages, however I don't think this is needed at all.

The only problem I see is when I drop a kernel series for given reasons. This will automatically block an update when still installed.

Sure it is easy to follow the Arch-Way, but then why should I bother to build our kernels? We have different features in them than upstream and it was good as it is now.

So only point to discuss would be meta packages and nothing more.

[Ste74]

Yep, this seem how arch work..

[ThanosApostolou]

Explicit kernel packages (e.g linux4.11) can still be built and included at repositories if some specific hardware requires only a specific kernel version (there shouldn't be any such cases normally), but they shouldn't be considered by the kernel pipe updating procedure and shouldn't be easily accessible to normal users but only for advanced ones.

Yeah I don't want to take away the ability to maintain specific kernel version. The point is that this shouldn't be visible to the simple users since 99% of the users shouldn't need a specific kernel version for their hardware to work. Also, 99% users should be updated to the next kernel version (linux-lts by default) without any intervention. I believe that same logic can be achieved with metapackages, as long as they are shown at the manjaro-settings kernel utility and they are installed by buildiso (linux-lts by default and selection of other kernel metapackages with -k option).

[ThanosApostolou]

Yeah, that's similar to what Arch, Solus and Tumbleweed are doing (and every other rolling distro as far as I know). It's just that those distros usually have linux-current and linux-lts. We can even add linux-prev to the pipe just below linux-current and this way Manjaro will have 5 different well supported and tested kernel series for their users to choose (which is great advantage). And then a top of that we can have 1 or 2 kernels out of the pipe like linux318 or linux411 just for the advanced users who know that their hardware can run better on those kernels (in my opinion these should be available packages but they shouldn't been shown in manjaro-settings kernel utility).

Also, as far as I know, future software can be broken with old kernels. For example mesa 17.2 has really bad performance with kernels older than 4.9 on certain hardware. Systemd may also need a kernel version or higher at future releases, etc... So, this kernel "pipe" achieves better testing and quality assurance, in my opinion.

This can be done with meta-packages too, if changing the naming is too much trouble. The significant change is what we mean by calling a kernel as supported in a rolling distribution. This shouldn't mean a fixed kernel version in my opinion, but more of a fixed updating schedule and testing waiting time (which differs if the choice is linux-current, linux-prev, linux-lts, etc...). Kernels with a fixed version should be available only for certain cases and be well documented as well.

That's my opinion anyway, tell me what you think :smiley:

EDIT - Updated diagram with linux-prev:

[Chrysostomus]

I think meta packages would be preferable, because it achieves the same goal without losing the flexibility of using different kernels. Downside is that it does not cut down the kernel maintenance for developers, upside is that it increases automation without sacrificing control for users.

However, as discussed on the corresponding forum thread, I think there might be issues with the module management with a simple meta package. Therefore I think that a post operation pacman hook (which calls mhwd-kernel) might be a good idea.

[oberon-manjaro]

I like the idea a lot! And yes, metapackages would be an easy way to achieve this. Like that all the kernel-pkgs themselves would just simply stay the same ane the named packages can just depend on the respective actual pkgs. So if someone needs a specific kernel for his hardware, they can still like now just install a cerain series and decide if and when to switch. For the average user @ThanosApostolou 's system will make things a lot easier to understand and handle. For us it's just a matter of adding a few extra meta-pkgs. They will just each have to be updated with their "real-life" kernel-pkgs...

[Ste74]

With the current way i only see a problem if a non lts kernel is eol.. i have ask at @kirek in the forum if msm can notify also for non lts kernel and it is not a problem... Msm also show the changelog as request...

[oberon-manjaro]

I think 'linux-prev' makes no sense, they can just be kept as normal packages and will not need a named meta-pkg. Regarding rt-kernels there is a little misunderstanding: We have linux-rt-lts-manjaro, which is the latest available rt-patch for an lts-kernel, and linux-rt-manjaro which is the latest available rt-patch. The patch for 4.13 was released just a few days ago and I will work on updating to 413_rt in the next days or so. So here the naming is kind of the other way round, because rt-users tend to want to have the newest available kernel - they typically use high-end hardware, anyway. So for me it still makes sense to keep the current naming for rt-kernels, linux-rt-manjaro being the (desired) newest one, and linux-rt-lts-manjaro the hopefully more reliable older one. In any case 'linux-lts-prev-rt' will not be a helpful lable 😉 Inspired by this discussion here I think I will definitely switch the rt-kernels to a meta-package system. This will make my life a LOT easier when rt-basekernels need to be switched - why didn't I think of that before!?? 😆

[ThanosApostolou]

Yeah I messed up the rt kernels in the diagram. Here's an updated diagram without the linux-prev.

Kernel meta-packages:

Kernel Update procedure in Pseudocode (I left out the rt kernels):

if ((upstream.mainline > linux-dev) && (linux-dev has at least 3 point releases)) {
    if (linux-current has been announced as lts) {
        linux-lts-prev=linux-lts
        linux-lts=linux-current
    }
    else {
        drop linux-current kernel build
    }
    linux-current=linux-dev
    linux-dev=upstream.mainline
}

NOTES:

1. upstream.mainline kernel is the mainline kernel as defined at https://www.kernel.org/. Kernel version is the major kernel series version (e.g. 4.14), point releases are get pushed as updates normally.
2. Maybe kernel metapackages should also have the -manjaro suffix (e.g. linux-dev -> linux-dev-manjaro) in order to match the rt kernel names.
3. (linux-dev has at least 3 point releases) condition can be modified, dependently on how stable we need linux-current or how fast we need linux-dev to reach the users (unstable branch).

[Bleuzen]

Any progress here? I think a linux-lts and linux-current meta-package would be great. It would make maintaining my machines easier so.. Manjaro Devs, have you already made a decision?

Sorry, if I'm too fast. I know, you need time to overthink such things. Just to remember, that the idea is not dead ;)

[Chrysostomus]

We'll, the decision has not been reached yet. Theoretically there should be decision on 1) should we do this? 2) which approach we should take? 3) who's going to do it?

In practice conversation goes nowhere, and things start to progress when someone decides to do something and write that meta-package/hook. After that conversation resumes, and if the implemented solution is something that doesn't cause extra work for others, it is likely to be accepted. Then it goes to unstable repos, gets tested, and once it has been in stable for some time, it gets considered for isos. Some community editions adopt it first, and of it seems not to cause trouble, it might get to official editions.

So, if someone finds time to do something, things start to progress. You can send a pull request if you want to hasten the process :)

[Ste74]

This is true but .. @philmmanjaro keeps all our main kernels so the first decision to invest on this is by Philip

[jonathonf]

I can see the point of having an linux-lts metapackage so that users who keep the default kernel don't have to think about it. User who manually install a newer kernel should know what they're doing, so they shouldn't really need a linux-latest.

The metapackage I wrote would do the job without very much faff.

---

Aside: why is development discussion split between GitHub and the forum?

[Ste74]

Aside: why is development discussion split between GitHub and the forum?

Good question 😉

[Bleuzen]

Yeah, I think we should continue the discussion here. I will post it in the forum, so that everybody knows it.

But anyway, sadly nothing seems to happen here :/

[jonathonf]

That's because this represents extra work with an unknown value. ;)

As @Ste74 already said, it's up to @philmmanjaro as to whether he wants to implement this in Manjaro (whether a metapackage or another solution).

[philmmanjaro]

To be frank: most likely not ...

[Ste74]

Reopen if needed