Support updating release notes in metainfo

[vchernin]

Today f-e-d-c can update the version number and release date in the metainfo/appdata file. I wonder if this could be extended to support adding the changelog details for each update to metainfo/appdata. This would require finding ways to parse an upstream changelog/news file. Relevant link

This would ideally improve the situation for many apps on Flathub (eg Chromium) that have a version history like this:

[wjt]

Yes, that would be great in theory. I fear it may be difficult to build something generic that can parse release information that is intended for end-users rather than developers.

Using Chromium as an example, is there an existing source that could work for this? The best I have found is "Stable Channel Update" posts on https://chromereleases.googleblog.com/

The alternative is for the human who merges each PR to add the news by hand.

[gasinvein]

Hmm, I as well doubt this could be possible in a generic way. Yet there are sources that provide release info we can grab, like release notes on GitHub. Maybe we should add a GitHubChecker?

[gasinvein]

Also, when/if we get either a working sandbox in github actions or a per-app f-e-d-c action, we can allow running custom scripts to retrieve release info during update by f-e-d-c.

[vchernin]

See also: https://github.com/ximion/appstream/issues/354

[nanonyme]

It would really be best approach to get as many projects as possible ship their appdata files that contain release notes. Core problem of this need to begin with is data that is supposed to come from upstream is getting patched together on Flathub side.

[vchernin]

I think perhaps more automated ways to maintain appdata release notes would help upstreams, https://blog.tenstral.net/2020/03/maintain-release-info-easily-in-metainfo-files.html helps but there are so many ways projects write release notes.

[nanonyme]

Does it btw support changelog? I noticed mosh actually has NEWS that is abandoned (?) and doesn't have newer releases, newer hings live in a file like https://github.com/mobile-shell/mosh/blob/master/ChangeLog.

[hadess]

FlashPrint has a manifest with the latest versions that the software pokes at http://www.ishare3d.com/3dapp/public/FlashPrint-5/appInfo1 which also links to a file with changelogs for the last few versions.

[gasinvein]

Finding an upstream changelog is usually easy - converting it to metainfo isn't. How can we parse a text changelog in arbitrary format? I personally doubt that it would be possible without each flatpak app downstream providing its own parser.

[hadess]

I personally doubt that it would be possible without each flatpak app downstream providing its own parser.

I would be fine with that, if updates start happening a bit too often... The main drawback is that you wouldn't want to run random software for the hosted flatpak-external-data-checker, it would need to run locally.

[nanonyme]

Would it make sense to patch the release notes in dynamically during app build and only add releases with f-e-d-c? Then the solution would be a lot more flexible. We could use metainfo as side-data for parsing release notes files.

[gasinvein]

Hmm, you mean grab upstream changelog with f-e-d-c at check time, commit it with update, and let the app handle it in its own way at built time?

[nanonyme]

Well, that is assuming changelog is present as file in release archive

[gasinvein]

@hadess Ok, then I suppose we should establish some CLI API to make this language-agnostic. I can think of two variants:

• After the manifest data is updated, but before metainfo.xml update, a script with well-known name gets executed with release parameters (e.g. version and date) on argv, which must spit out a <release> node for given parameters
• Same, but the script gets fed with the current metainfo.xml to stdin with <release> already added, and must spit out the whole file, altered the way it likes

[hadess]

The former is what would be interesting to me. There's nothing particularly complicated to mangling the upstream changelog, feeding it back into the existing metainfo file is what's more complicated from a casual standpoint.

[gasinvein]

@wjt Do you think we can still keep original indentation if we use <release> node produced by an external script?

[wjt]

Maybe -- the approach would presumably vary depending on whether the stanza to add to the file is treated as raw text or parsed as XML.

Maybe it's not so important.

[gasinvein]

@wjt Can we just always re-indent the metainfo.xml on write instead of trying to preserve original indentation? Probably taking editorconfig settings into account.

[razzeee]

If we can figure this out, this would IMO be a big win, currently, I end up touching each automatic update to add changelogs again.

[nanonyme]

The correct fix is upstream adds release notes into their appstream file. Are you the upstream?

[razzeee]

No, not in the projects I have this "problem" with. They all have their appstream data in the flathub repo

[nanonyme]

Note that you could also replace appstream file with template and add release notes during app build.

[razzeee]

Can we maybe implement something simple optional? Like a configuration, from where to grab the changelog and a regex, that get's used to find the relevant part. Maybe an additional regex to clean up unwanted markdown etc.

IMO we will continue to suffer from waiting for the perfect solution otherwise.