Maintain (active) releases in machine readable form

What would you like to be added: The linux kernel maintains a file at https://www.kernel.org/releases.json where releases are present in a machine-readable form. This is pretty nice for automation purposes. We could do something similar, creating such a file could be done by our release-notes workflow.

Why is this needed: In automation, we sometimes need to know (maintained) Garden Linux versions. We can use git tags or github releases in this repo, but this is missing some information/meta data.

This could be useful to provide a similar view to https://packages.debian.org/search?suite=all&searchon=names&keywords=wget where we want to know the package versions present in (relevant) Garden Linux versions.

This could also be useful for GLVD.

Just a rough sketch of what this file could look like:

{
    "releases": [
        {
            "moniker": "something-needs-to-be-aligned",
            "version": "1592.1",
            "iseol": false,
            "released": {
                "timestamp": 1724569631,
                "isodate": "2024-08-25"
            }
        },
        {
            "moniker": "something-needs-to-be-aligned",
            "version": "1592.0",
            "iseol": true,
            "released": {
                "timestamp": 1724569631,
                "isodate": "2024-08-25"
            }
        },
        {
            "moniker": "something-needs-to-be-aligned",
            "version": "1443.10",
            "iseol": false,
            "released": {
                "timestamp": 1724569631,
                "isodate": "2024-08-25"
            }
        }
  ]
}

@yeoldegrove would that help you instead of :latest cat /etc/os-release?

@pnpavlov @fwilhe

https://github.com/gardenlinux/gardenlinux/commit/87e2dc920e906345d19b19a40c78b08d5af8e389#diff-9dd0c2a02c4e49d2c385f2033fa07902c78b11ef7013e8fbfbf90d19d6e6057c would be a quick hack to parse ghcr.io/gardenlinux/gardenlinux:latest and extract the actual release tag.

❯ bin/garden-version-latest
1592.1

@yeoldegrove , please check if this is in same direction as your current work? Can we do that?

Alternative suggestion for file layout

The idea here would be to group releases by major versions (one new every day) and to mark which of those are 'maintained', i.e. get any patch releases. I'm not sure if this adds much value, or if we should leave out those nightly releases.

{
    "releases": [
        {
            "major": "1602",
            "isMaintained": false,
            "patches": []
        },
        {
            "major": "1592",
            "isMaintained": true,
            "patches": [
                {
                    "version": "1592.1",
                    "iseol": false
                },
                {
                    "version": "1592.0",
                    "iseol": true
                }
            ]
        },
        {
            "major": "1443",
            "isMaintained": true,
            "patches": [
                {
                    "version": "1443.10",
                    "iseol": false
                },
                {
                    "version": "1443.9",
                    "iseol": true
                }
            ]
        }
    ]
}

Alternatively it might also look like this where we add only the maintained major versions. The individual objects should each have more information than in this example, i.e. release date, etc.

{
    "releases": [
        {
            "major": "1592",
            "patches": [
                {
                    "version": "1592.1",
                    "iseol": false
                },
                {
                    "version": "1592.0",
                    "iseol": true
                }
            ]
        },
        {
            "major": "1443",
            "patches": [
                {
                    "version": "1443.10",
                    "iseol": false
                },
                {
                    "version": "1443.9",
                    "iseol": true
                }
            ]
        }
    ]
}

those suggestions leave out qualifiers like 'longterm' etc, I don't really think this makes sense in our case, but I'm open to challenge this.

After an initial file was hand-crafted, updates of this file should be possible to do fully automated in our ci pipeline. The needed information should be available there.

for the SapMachine use-case it is required to be able to understand:

what are current releases
which is the latest release
is the release in maintenance

fmi Christian (SapMachine)

After thinking a bit about what we currently have defined in https://github.com/gardenlinux/gardenlinux/blob/main/docs/00_introduction/release.md and what was suggested by Florian, I would suggest something like this...

A section called releases for each official released patch version.
- can be initially scraped from https://github.com/gardenlinux/gardenlinux/releases
lifecycle.released is actual release date
lifecycle.eol is:
- if latest minor --> taken from releases_stable.${major}
- if not latest minor --> taken from releases.${major}'s latest minor entry
additional fields could be added for:
- download addresses for images
- might also be able to generate those from S3+${version}+${commit}
additional fields should not added for
- isEOL or isMaintained
- We can stick to evaluating timestamps.

releases:
- name: release-1443.10
  version:
    major: 1443
    minor: 10
  lifecycle:
    released:
      isodate: '2024-07-24'
      timestamp: 1721772000
    eol:
      isodate: '2025-01-13'
      timestamp: 1736722800
  git:
    commit: 8d098305ade0addbe03be324cc50b1bf4bf3b206
    commit_short: 8d09830
  github:
    release: https://github.com/gardenlinux/gardenlinux/releases/tag/1443.10
- name: release-1592.0
  version:
    major: 1592
    minor: 0
  lifecycle:
    released:
      isodate: '2024-08-12'
      timestamp: 1723413600
    eol:
      isodate: '2024-08-22'
      timestamp: 1724277600
  git:
    commit: 0cc2b85de0e70a44c12dd861f2cbce8d395a2255
    commit_short: 0cc2b85
  github:
    release: https://github.com/gardenlinux/gardenlinux/releases/tag/1592.0
- name: release-1592.1
  version:
    major: 1592
    minor: 1
  lifecycle:
    released:
      isodate: '2024-08-22'
      timestamp: 1724277600
    eol:
      isodate: '2025-08-12'
      timestamp: 1754949600
  git:
    commit: ec945aa995d0f08d64303ff6045b313b40b665fb
    commit_short: ec945aa
  github:
    release: https://github.com/gardenlinux/gardenlinux/releases/tag/1592.1

A section called releases_stable
- lifecycle.extended and lifecycle.eol have to be manually defined/updated

releases_stable:
- name: release-1443
  version: 1443
  lifecycle:
    released:
      isodate: '2024-03-13'
      timestamp: 1710341636
    extended:
      isodate: '2024-09-13'
      timestamp: 1726178400
    eol:
      isodate: '2025-01-13'
      timestamp: 1736722800
- name: release-1592
  version: 1592
  lifecycle:
    released:
      isodate: '2024-08-12'
      timestamp: 1723457202
    extended:
      isodate: '2025-05-12'
      timestamp: 1747000800
    eol:
      isodate: '2025-08-12'
      timestamp: 1754949600

With the data from https://github.com/gardenlinux/gardenlinux/releases and some manual inputs from https://github.com/gardenlinux/gardenlinux/blob/main/docs/00_introduction/release.md one could easily come up with some tool like this:

❯ bin/garden-releases --input releases.json --output-type shell
Name                    Version     Git Commit  Release date    Extended maintenance    End of maintenance  
release-27              27.0        N/A         2020-06-09      2020-12-09              2021-03-09          
release-27.0            27.0        504eae6     2020-06-09      N/A                     2020-06-17          
release-27.1            27.1        7a36937     2020-06-17      N/A                     2021-03-09          
release-184             184.0       N/A         2020-10-29      2020-04-29              2021-07-29          
release-184.0           184.0       e5a1576     2020-10-29      N/A                     2021-07-29          
release-318             318.0       N/A         2021-04-28      2021-10-28              2023-01-28          
release-318.4           318.4       36317a7     2021-04-28      N/A                     2021-06-24          
release-318.8           318.8       f5706b8     2021-06-24      N/A                     2021-10-13          
release-318.9           318.9       7dfdb57     2021-10-13      N/A                     2023-01-28          
release-576             576.0       N/A         2021-11-17      2023-05-17              2023-08-17          
release-576.0           576.0       d3c7f24     2021-11-17      N/A                     2021-12-09          
release-576.1           576.1       0f7a5b0     2021-12-09      N/A                     2022-01-05          
release-576.2           576.2       9e22b9d     2022-01-05      N/A                     2022-01-26          
release-576.3           576.3       c7bb2e0     2022-01-26      N/A                     2022-02-23          
release-576.5           576.5       634ba8e     2022-02-23      N/A                     2022-03-18          
release-576.7           576.7       7d3c7d9     2022-03-18      N/A                     2022-04-19          
release-576.8           576.8       7c12cd7     2022-04-19      N/A                     2022-05-12          
release-576.9           576.9       229156b     2022-05-12      N/A                     2022-06-20          
release-576.10          576.10      5187558     2022-06-20      N/A                     2022-08-03          
release-576.11          576.11      80ca440     2022-08-03      N/A                     2022-09-09          
release-576.12          576.12      b10a46f     2022-09-09      N/A                     2023-08-17          
release-934             934.0       N/A         2023-06-05      2023-12-05              2024-03-05          
release-934.0           934.0       86f69dc     2023-06-05      N/A                     2022-11-11          
release-934.1           934.1       b1021e6     2022-11-11      N/A                     2023-01-10          
release-934.2           934.2       d693dc1     2023-01-10      N/A                     2023-01-26          
release-934.3           934.3       1e14c6e     2023-01-26      N/A                     2023-02-17          
release-934.4           934.4       07005e4     2023-02-17      N/A                     2023-03-02          
release-934.5           934.5       b8e05d7     2023-03-02      N/A                     2023-03-14          
release-934.6           934.6       76ea662     2023-03-14      N/A                     2023-03-22          
release-934.7           934.7       de96b16     2023-03-22      N/A                     2023-05-10          
release-934.8           934.8       d959891     2023-05-10      N/A                     2023-06-02          
release-934.9           934.9       54c63e5     2023-06-02      N/A                     2023-08-16          
release-934.10          934.10      f057c9b     2023-08-16      N/A                     2023-10-25          
release-934.11          934.11      75132b8     2023-10-25      N/A                     2024-03-05          
release-1312            1312.0      N/A         2023-11-16      2023-05-03              2024-08-03          
release-1312.0          1312.0      40b9db2     2023-11-16      N/A                     2023-11-23          
release-1312.1          1312.1      2a12736     2023-11-23      N/A                     2024-01-15          
release-1312.2          1312.2      177c5ce     2024-01-15      N/A                     2024-02-14          
release-1312.3          1312.3      367f95e     2024-02-14      N/A                     2024-06-14          
release-1312.5          1312.5      9bd6f63     2024-06-14      N/A                     2024-07-03          
release-1312.7          1312.7      330e891     2024-07-03      N/A                     2024-08-03          
release-1443            1443.0      N/A         2024-03-13      2024-09-13              2025-01-13          
release-1443.0          1443.0      ea851d6     2024-03-13      N/A                     2024-03-25          
release-1443.1          1443.1      a84c44a     2024-03-25      N/A                     2024-04-02          
release-1443.2          1443.2      5e25991     2024-04-02      N/A                     2024-04-11          
release-1443.3          1443.3      c261f88     2024-04-11      N/A                     2024-05-23          
release-1443.5          1443.5      bfb687a     2024-05-23      N/A                     2024-06-19          
release-1443.7          1443.7      34b0238     2024-06-19      N/A                     2024-06-25          
release-1443.8          1443.8      5a31e92     2024-06-25      N/A                     2024-07-03          
release-1443.9          1443.9      964570b     2024-07-03      N/A                     2024-07-24          
release-1443.10         1443.10     8d09830     2024-07-24      N/A                     2025-01-13          
release-1592            1592.0      N/A         2024-08-12      2025-05-12              2025-08-12          
release-1592.0          1592.0      0cc2b85     2024-08-12      N/A                     2024-08-22          
release-1592.1          1592.1      ec945aa     2024-08-22      N/A                     2025-08-12        

❯ bin/garden-releases --input releases.json --output-type shell --latest
Name                    Version     Git Commit  Release date    Extended maintenance    End of maintenance  
release-1592.1          1592.1      ec945aa     2024-08-22      N/A                     2025-08-12    

❯ bin/garden-releases --input releases.json --output-type shell --active
Name                    Version     Git Commit  Release date    Extended maintenance    End of maintenance  
release-1443            1443.0      N/A         2024-03-13      2024-09-13              2025-01-13          
release-1443.10         1443.10     8d09830     2024-07-24      N/A                     2025-01-13          
release-1592            1592.0      N/A         2024-08-12      2025-05-12              2025-08-12          
release-1592.1          1592.1      ec945aa     2024-08-22      N/A                     2025-08-12

Instead of having a split between releases_stable and releases, one could also define type: 'stable' and type: 'patch'.

Hi @yeoldegrove, I think this looks good 👍

Things to note: We do sometimes 'burn' patch versions, i.e. that version 1443.7 exist does not imply that version 1443.6 exists. It might or it might not.

For this reason, I'd like to have a complete list of patch releases in that release file. I don't think this contradicts what you said, just to make this explicit.

I'm not sure how certain we are with those release and support timeframes, this falls into the responsibility of @pnpavlov. Would you be fine with this?

For GLVD specifically, what I want are mostly two things:

I'd like to have an automated way to know what the 2 or 3 or 4 current versions are (i.e. the latest patch release of the last few major releases). I think this is covered by your suggestion. Those should be prominently featured in the GLVD user interface.

Second, for all released versions (this excludes the 'burned' patches mentioned above), I need a way to construct the url for getting the list of packages. Essentially all information i need for that is a list of version numbers, so a structure like 1443: [1443.0, 1443.1, 1443.2 ..], 1592: [1592.0, 1592.1, ..] would be good enough for that.

This extended life-cycle information might also be useful, for example the GLVD cli tool or web ui might be able to warn users about 'you are using version that does not get any security fixes, please upgrade asap'. I think this is useful, if we have a meaningful definition of those support timeframes.

@fwilhe If you burn a patch version, is it supposed to be found later or is it supposed to "just not exist"? In the later case the patch release just has to be removed and it will never be found again. Another possibility to flag this as burned would be setting the eol date to the same date as the release date or deleting the release date.

I am still not fond of the idea to reference the patch versions in the stable version's entry (e.g. having a version.patches: [1443.0, 1443.1, 1443.2]) field. The whole point of having a naming scheme like ${major}.${minor} is that it is clear which version is the parent. This does not need a reference field or are there any good arguments for this? If I want to have all patches for a release I can build that filter/query into the client that queries the "release database".

This is my latest take on how to define releases and their schemas. Based on the discussions above and our current documentation.

Releases in general (from https://github.com/gardenlinux/gardenlinux/blob/main/docs/00_introduction/release.md)

You can expect up to four Garden Linux releases per year, published up to once per quarter. A new release (e.g. 1443.0 or "release 1443") includes major changes like newer major version of software package or library. Within a release maintenance period (release date -> End-of-life date), changes are published as release patches (e.g. 1443.1 or "patch 1 of release 1443"). Each release is maintained nine months, while going through following maintenance phases:

Standard maintenance starts with the publishing of a new stable release and continues for six months. In this phase active maintenance is performed on the release. You can expect regular patches including (but not limited to) fixes for issues, updates to higher minor release of selected software packages and libraries included, proactive monitoring & assessment of security vulnerabilities, performance corrections and enhancements and more. General best practice is to always use the latest patch of the latest release.
Extended maintenance starts after the standard maintenance phase and continues for three months. In this phase only Common Vulnerabilities and Exposures (CVE) with high and critical (7.0-10.0) severity according to the Common Vulnerability Scoring System (CVSS) are patched. For more details refer to the Garden Linux Security Response Process. This enables users to have sufficient time to plan and execute their transition to latest release in standard maintenance.
End of maintenance is reached nine months after the release has been published. In this phase no further planned maintenance activities are performed. Adopters can continue to use the release at their own risk. Users can request an exceptional patch via GitHub issue in gardenlinux. There is no guarantee the request will be served - maintainers reserve the right to reject the request and fix-forward only in the latest release that is in standard maintenance due to the complexity involved with producing a patch for very old release.

Release Schema

The Garden Linux Release Database (GLRD) uses structured JSON schemas to represent different types of releases: stable, patch, nightly, and development releases. Each release type has specific fields that capture essential information about the release.

Stable Releases

Stable releases are major releases that are supported over an extended period of time (see above).

Schema Fields

name: A string representing the release name (e.g., release-1312).
type: stable.
version:
- major: An integer indicating the major version number (e.g. 1312).
lifecycle:
- released:
- isodate: The release date in ISO format (YYYY-MM-DD).
- timestamp: The UNIX timestamp of the release date.
- extended:
- isodate: Optional extended maintenance date in ISO format.
- timestamp: Optional UNIX timestamp for the extended maintenance date.
- eol:
- isodate: End-of-life date in ISO format.
- timestamp: UNIX timestamp for the end-of-life date.

Patch Releases

During the standard and extended maintenance periods the corrections are delivered via patch releases ("patches"). They are considered stable, and adopters are encouraged to always use the latest patch of the latest stable release. Patch releases can be expected to adhere to following rules:

Can include higher minor versions of software packages or libraries already included in the LTS release.
Can include security enhancements related to vulnerabilities identified after the release of the previous patch.
Can remove in rare (exceptional) situations software packages or libraries (e.g. due to security or legal risk identified after the release was published.). Such changes are documented in the release note.

Schema Fields

name: A string representing the release name (e.g., release-1312.1).
type: patch.
version:
- major: An integer indicating the major version number (e.g. 1312).
- minor: An integer indicating the minor version number (e.g. 1).
lifecycle:
- released:
- isodate: The release date in ISO format.
- timestamp: The UNIX timestamp of the release date.
- eol:
- isodate: End-of-life date in ISO format.
- timestamp: UNIX timestamp for the end-of-life date.
git:
- commit: The full git commit hash associated with the release.
- commit_short: The short git commit hash (first 7 characters).
github:
- release: The URL to the GitHub release page.

Nightly Releases

Nightly releases are automated builds that are generated every night, reflecting the latest state of the codebase.

Schema Fields

name: A string representing the release name (e.g., nightly-1312.0).
type: nightly.
version:
- major: An integer indicating the major version number.
- minor: An integer indicating the minor version number.
lifecycle:
- released:
- isodate: The release date in ISO format.
- timestamp: The UNIX timestamp of the release date.
git:
- commit: The full git commit hash associated with the release.
- commit_short: The short git commit hash.

Development Releases

Development releases are used for testing and development purposes, representing the latest changes that may not yet be included in a stable or patch release. These can be manually created by developers.

Schema Fields

name: A string representing the release name (e.g., dev-1312.0).
type: dev.
version:
- major: An integer indicating the major version number.
- minor: An integer indicating the minor version number.
lifecycle:
- released:
- isodate: The release date in ISO format.
- timestamp: The UNIX timestamp of the release date.
git:
- commit: The full git commit hash associated with the release.
- commit_short: The short git commit hash.

Lifecycle Dependencies

The lifecycle fields in the release schemas help track the release dates and end-of-life (EOL) dates for each release. There is a dependency between the lifecycle fields of stable and patch releases:

EOL of Patch Releases: The eol date of a patch release is set to the released date of the next patch release. If there is no subsequent patch release, the eol date is aligned with the eol date of the corresponding stable release.
EOL of Latest Patch Release: The latest patch release's eol date matches the eol date of the stable release.
EOL of Stable Releases: The eol date for a stable release marks the end of support for that major version.

This ensures that all minor updates within a major release adhere to the same overall support timeline.

For example:

Stable Release release-1312:
- released: 2023-11-16
- extended: 2024-05-03
- eol: 2024-08-03
Patch Release release-1312.1:
- released: 2023-11-23
- eol: 2024-01-15 (next patch release date)
Patch Release release-1312.2:
- released: 2024-01-15
- eol: 2024-02-14 (next patch release date)
...
Patch Release release-1312.7:
- released: 2024-07-03
- eol: 2024-08-03 (inherits from stable release eol)

In this example, the eol of release-1312.1 is set to the released date of release-1312.2, and the eol of the latest patch release (release-1312.7) is set to the eol of the stable release (release-1312).

❯ ./glrd --type stable,patch --version 1312
Name                    Version                 Type                    Git Commit              Release date            Extended maintenance    End of maintenance  
release-1312            1312                    stable                  N/A                     2023-11-16              2024-05-03              2024-08-03          
release-1312.0          1312.0                  patch                   40b9db2                 2023-11-16              N/A                     2023-11-23          
release-1312.1          1312.1                  patch                   2a12736                 2023-11-23              N/A                     2024-01-15          
release-1312.2          1312.2                  patch                   177c5ce                 2024-01-15              N/A                     2024-02-14          
release-1312.3          1312.3                  patch                   367f95e                 2024-02-14              N/A                     2024-06-14          
release-1312.5          1312.5                  patch                   9bd6f63                 2024-06-14              N/A                     2024-07-03          
release-1312.7          1312.7                  patch                   330e891                 2024-07-03              N/A                     2024-08-03

Please note that the extended lifecyle field is not taken into account for patch releases. This is simply an administrative date that has no technical implications.

https://github.com/gardenlinux/glrd hosts my proposal for a Garden Linux Release Database (GLRD).

https://github.com/gardenlinux/gardenlinux/pull/2409 enables GLRD for nightly and stable/patch releases.

gardenlinux / gardenlinux