Changelog info text in F-droid

[aha999]

Use case

Currently, there is no visible window with the latest changes when you open your app's page in F-Droid.

Proposed Solution

Add changelog text when you open the fdroid app repo

Some tips:

F-Droid looks for metadata in the app’s source repo. The supported formats for this metadata are borrowed from popular projects that automate uploading this metadata to Google Play Store.

The one in your example corresponds to Fastlane Supply and is documented here. All formats supported by F-Droid are listed here

[matkoniecz]

I think that it is a bug caused by F-Droid and should be reported on F-Droid repository.

Is there some reason to suspect that it is bug caused by StreetComplete itself? This works well when installed from Google Play, it works well when sideloaded.

Is F-Droid modifying StreetComplete in some way before releasing it?

[aha999]

I think that it is a bug caused by F-Droid and should be reported on F-Droid repository.

Maybe @eighthave would know how to answer this ?

Is F-Droid modifying StreetComplete in some way before releasing it?

I don't think so. They build straight from source AFAIK.

[westnordost]

The changelog is shown within the app. It's not shown on F-Droid.

To enable showing the changelog in F-Droid, we'd need to put the changelogs for the different versions in \metadata\en\changelogs\<version_code>.txt (it's only available in English).

To do this automatically is non-trivial because:

1. A gradle task would need to parse this file https://github.com/streetcomplete/StreetComplete/blob/master/app/src/main/res/raw/changelog.yml and take it apart - create one file for each version.
2. However, the filename of the changelog entries in \metadata\en\changelogs\<version_code>.txt must be the version code (e.g. 3501), not the version name (v35.0). I don't see a clear method how we can get the information which version name belongs to wich version code.

So, if there are no ideas for this, I'll close this as will not fix.

[FloEdelmann]

We could do it the other way around: Manually create one file per version in that location, and then combine them into the all-in-one file with a Gradle task.

(The Gradle task would need to exclude all beta versions though, except if there is no newer stable version.)

[westnordost]

And how to get from version code to version name then?

[FloEdelmann]

Just leave it as a HTML heading in the individual files.

[westnordost]

Hmm, that would be possible. I wonder what F-Droid does with *.txt files that are actually html (snippets) though. We surely don't want all the HTML be displayed as plain text.

[peternewman]

2. However, the filename of the changelog entries in \metadata\en\changelogs\<version_code>.txt must be the version code (e.g. 3501), not the version name (v35.0). I don't see a clear method how we can get the information which version name belongs to wich version code.

Can you pull the current version out via Gradle? Then if it was one monolithic file it could just write the most recent bit to the relevant new changelog; older changelogs generally shouldn't change and it won't care if they did.

Hmm, that would be possible. I wonder what F-Droid does with *.txt files that are actually html (snippets) though. We surely don't want all the HTML be displayed as plain text.

Or work on that idea/the original one and add some more fields to the YAML:

v36.0:
  code: 3601
  log: |
  <h3>Easier orientation</h3>
  <p>🖊️ The path you went today with the app is now marked on the map for better orientation! (#2209)</p>
  <p>The location button (lower right corner) also has a new behavior:</p>
  <ul>
  <li>🛰️ Tap it once to let the view follow your position on the map (icon turns orange)</li>
  <li>🚴 Tap it again to let the view rotate with the direction you are going (icon turns into an
      arrow). Very helpful for cyclists! (#1040, #2191, #3335)</li>
  </ul>
  <p>Furthermore, the only thing the compass button now does is to rotate the view back to north-up
     and it is only displayed if the map is not displayed north-up.
  </p>
  <h3>New Quests</h3>
  <ul>
  <li><i>Is this picnic table covered (protected from rain)?</i> (#3311, #3142), by @FloEdelmann</li>
  <li><i>Does ... offer halal products?</i> (#2990, #3331), by @SMUsamaShah</li>
  </ul>
  <h3>Quest Enhancements</h3>
  <ul>
  <li>Add answer option to state that there are actually steps, not a footway (#3288, #3334), by @FloEdelmann</li>
  <li>Building type: Add building bridge (skyway) as answer option (#2630, #3320), by @kmpoppe</li>
  <li>Self-service laundry: Add answer option to say self-service is optional (#3309)</li>
  <li>Don't ask some of the quests for private things (#3356, #3357, #3358), by @dbdean</li>
  </ul>
  <h3>Other enhancements, fixes, maintenance</h3>
  <ul>
  <li>More contrast on text (#3328), by @matkoniecz</li>
  <li>(#3327) by @FloEdelmann, (#3208) by @matkoniecz</li>
  </ul>

Then we could e.g. strip the HTML (although maybe that should go upstream), generate multiple files, drop emoji etc as required for the various outputs.

[westnordost]

Mh well, that would be possible. @aha999, do you have any information on how F-Droid treats HTML? Or if it supports HTML, markup or something else at all beside .txt?

[westnordost]

Also, do you know an example app for which this changelog stuff is used? I would like to see how it is displayed because I am still not sure if this is worth it at all.

[riQQ]

AntennaPod has information about changes in the latest version.

[mnalis]

@westnordost Relatively simple example might be https://f-droid.org/en/packages/org.woheller69.eggtimer/ which has a grey box which says:

New in version 1.4 Optional selection of weight instead of weight category Additional options for start temperature Link to project on GitHub Bugfixes

that grey box on web is also what is displayed in f-droid app like this:

Text for changelog seems to come from here: https://github.com/woheller69/eggtimer/blob/main/fastlane/metadata/android/en-US/changelogs/

[FloEdelmann]

HTML Markup seems to be possible in at least the app description: https://forum.f-droid.org/t/f-droid-support-fastlane-full-description-in-html-or-md-instead-of-txt/9395

[smichel17]

Alternate link: https://f-droid.org/en/docs/All_About_Descriptions_Graphics_and_Screenshots/#html-descriptions

I don't know if html is supported in the changelog. I did a quick search for the relevant code but all I turned up was this. If it is, it doesn't matter what file you put it in; it's all loaded into the relevant part of the listing the same way (source: code — search case insensitive for whatsnew).

Note: I'm planning to do metadata-related work on F-Droid (fdroid/fdroidserver#829). So, if you can't figure out a good workflow with the current situation, "request changes upstream" is a more realistic option than normal, since I'll be familiar with the relevant code.

[westnordost]

What exactly are you planning to do and why?

[smichel17]

Oop sorry, I forgot to delete a part that I re-wrote before posting. Edited that part out, it should be clearer now. More details:

What: Right now, F-Droid supports two metadata formats: Fastlane (what StreetComplete uses) and Triple-T. Both suck: Fastlane can't do build flavor specific metadata, and Triple-T only stores the most recent changelog, so there's no way to re-use it in the app. I'm allowing mixing and matching the good parts from each.

## Specifics - **Changelog Format:** Fastlane uses separate txt files corresponding to the version code. Triple-T uses a single `whatsnew` file which must be overwritten for each release. - **Fastlane** gets this right. With Triple-T's approach, the history is only in git, so there's no easy way to re-use this data for display in the app. - **Folder location:** Fastlane uses a `metadata/` folder in the repo root. Triple-T uses `/src//play/` (e.g. `app/src/main/play/`), - **Triple-T** gets this right; it supports multiple build flavors and multiple apps per repo. Build flavors is especially relevant since many apps have separate flavors for F-Droid and the Play Store. - **Folder structure (locales):** Too many differences to list, see [here](https://f-droid.org/en/docs/All_About_Descriptions_Graphics_and_Screenshots/#fastlane-structure) for Fastlane and then scroll down for Triple-T. - **Neither** is objectively better. Personally I think Fastlane is a little less awkward, with fewer unnecessary folders. Right now, F-Droid looks for a Triple-T structured metadata folder in the Triple-T location, then does the same for Fastlane. I'm changing it so it will look in both locations for a folder with either structure, so you can use Fastlane-style changelogs in the Triple-T location. ---

Why: I am in search of a less painful changelog release process. Specifically, I'd like the app and F-Droid to pull changelogs from the same location. I don't think these changes will allow that on their own, but they're fully backwards compatible, so they're good first steps.

The important part: When trying to figure out your options, you don't have to limit yourself to how StreetComplete can work around F-Droid's current deficiencies; you can also figure out how you'd like F-Droid to work and we can talk about whether that's possible.

However, this has been on my list for ~a year, mostly because I have little patience for wrestling with python tooling. If anyone wants to help me get to that faster, the most useful thing would be some help getting a dev environment set up so I can actually test the damn changes I'm making.

[matkoniecz]

I am confused: is this issue about something actionable on StreetComplete side, given current state of supported formats?

you can also figure out how you'd like F-Droid to work

seems to indicate that it is about F-droid redesign what would fit better to issue/discussion on F-Droid repository wherever it is

[smichel17]

What I linked to is entirely upstream, not actionable on SC side. I only intended to make you aware of it, so you'd know that requesting changes upstream is a reasonable thing in this case, if you can't figure out a good workflow for changelogs— which I find very likely, because the current process sucks.

I think this was mostly confusing because I accidentally posted an irrelevant part of a sentence, which I intended to delete. I've now edited my comments above substantially and hidden all comments about F-Droid changes. Please let me know if it's still confusing.

[westnordost]

Cool that you are working on (improving) F-Droid!

This ticket however I think I'll close. The changelog is already shown in-app, a place many more people will likely read. The way it is now, it is not possible without a lot of effort to generate the fastlane-changelog txts automatically.

[westnordost]

That having said, if someone would like to do something like this, you are invited to do so. However:

• adding things to the changelog shall not be more work than now
• changelog should be writable in simple HTML or markdown
• the source changelog file should be in the resources of the app itself and the data in metadata should only be generated from that and not the other way round
• such jobs should be implemented as gradle tasks, not python or something like this

[smichel17]

the source changelog file should be in the resources of the app itself and the data in metadata should only be generated from that and not the other way round

This is the part that requires changes to F-Droid. It currently won't look inside the app resources for metadata.

About the current format, though: the most annoying part of keeping https://github.com/streetcomplete/StreetComplete/pull/2794 up to date (I merged master several times) was dealing with merge conflicts in changelog.yml. I think one file per release would have made that easier (and would be more compatible with F-Droid, in the future).

changelog should be writable in simple HTML or markdown

Imo markdown is the best option here. It's the easiest to update and copy-paste to different places (e.g. GitHub releases) as needed, and it looks fine in plaintext. I don't think there's a good markdown renderer for Android, though. They all render to html and then use a webview; at that point I think it should be a build step instead of bundling another dependency in the apk.

[westnordost]

Imo markdown is the best option here. It's the easiest to update and copy-paste to different places (e.g. GitHub releases)

This

is HTML and it

• works

no problem

[mnalis]

GitHub seems to support very limited HTML subset; mostly what is already provided by Markdown? In which case, it might be preferable to stick to Markdown as it is, as it is also human-readable when rendered as regular non-formatted text (as opposed to HTML)

For example: this should be green text in HTML, and those are

big letters.

And also this nice button: is invisible in GitHub, this progressbar is not working, both Checked and unchecked checkboxes are not visible, etc.

source

```html For example: this should be green text in HTML, and those are

big letters.

And also this nice button: is invisible in GitHub, this progressbar is not working, both Checked and unchecked checkboxes are not visible, etc. ```

[westnordost]

If the changelog was in markdown, we'd need to include a library that can display a markdown-formatted string. Also, someone would need to convert the current changelog to markdown. Currently, we use a library to display simple html.

[smichel17]

I take back what I said about there being no Android library to render markdown, that was several years ago but in the meantime it looks like https://github.com/noties/Markwon has been developed.

[peternewman]

the most annoying part of keeping #2794 up to date (I merged master several times) was dealing with merge conflicts in changelog.yml.

OT but you failed @smichel17 https://github.com/streetcomplete/StreetComplete/blob/7519268ad4499ca65037f706b22c732b2316627c/app/src/main/res/raw/changelog.yml#L162

I can't edit that branch directly so I'll just leave a comment here...

I think one file per release would have made that easier (and would be more compatible with F-Droid, in the future).

Given the version for your change appears to have kept changing I'm not convinced it would solve the merge issue.

[smichel17]

I'm not convinced it would solve the merge issue.

It would not prevent merge conflicts, but it would make those conflicts much easier to resolve by preventing git from mixing the sections. This is really a shortcoming of git that we're working around, but I'm not going to recommend that we switch to a different VCS :P

[mnalis]

It would not prevent merge conflicts, but it would make those conflicts much easier to resolve by preventing git from mixing the sections. This is really a shortcoming of git that we're working around, but I'm not going to recommend that we switch to a different VCS :P

I agree, associativity is a problem in git, and in changelog example in particular splitting one-version-per-file would help a lot.

Regarding different VCS (offtopic), any VCS whose docs explain the things with:

With branches, the first scenario looks like this: Bob creates A, while Alice creates B

is doomed to fail :laughing:

[aha999]

Mh well, that would be possible. @aha999, do you have any information on how F-Droid treats HTML? Or if it supports HTML, markup or something else at all beside .txt?

i do not. I only know that a lot of projects use this feature to have changelogs displayed but if other formats are allowable i would not know that. Maybe @eighthave has an answer to this ?

[tiptoptom]

Do I understand correctly that now that we have the changelog as .md, we only need the version number and could then build an automatism?

[FloEdelmann]

Yeah, basically. But linking each version code to a Markdown heading might not be that easy.

[tiptoptom]

Unfortunately, I really don't know how this works with the tasks. Where is the version code defined? And if you have the version code and version name, can't you regex it?

Alternatively, if that makes it easier, I think you could always copy the entire changelog and add the current version code.
https://github.com/M66B/FairEmail/tree/master/metadata/en-US/changelogs

[FloEdelmann]

I really don't know how this works with the tasks

They are defined in https://github.com/streetcomplete/StreetComplete/tree/master/buildSrc and registered in https://github.com/streetcomplete/StreetComplete/blob/master/build.gradle.kts and https://github.com/streetcomplete/StreetComplete/blob/master/app/build.gradle.kts. You can execute them with e.g. ./gradlew generateQuestList or ./gradlew test.

Where is the version code defined?

In f7c0cc61/app/build.gradle.kts#L37. Since there is also the version name there, matching could indeed be feasible.

And if you have the version code and version name, can't you regex it?

Yeah, that's basically what happens in the changelog dialog in the app as well, see f7c0cc61/app/src/main/java/de/westnordost/streetcomplete/screens/about/ChangelogFragment.kt#L56-L64.

[scollovati]

For some reasons here it appears: https://f-droid.org/en/packages/de.westnordost.streetcomplete.expert/ instead in the basic version does not work: https://f-droid.org/en/packages/de.westnordost.streetcomplete/

[riQQ]

For SCEE 56.0 there was no changelog. So this must be something recent. See archived page here: https://web.archive.org/web/20240202105029/https://f-droid.org/en/packages/de.westnordost.streetcomplete.expert/

[Helium314]

You can see SCEE versions that have a changelog here: https://github.com/Helium314/SCEE/tree/modified/metadata/en-US/changelogs

[scollovati]

You can see SCEE versions that have a changelog here: https://github.com/Helium314/SCEE/tree/modified/metadata/en-US/changelogs

Exactly. I was wondering why the base project do not make it available for F-Droid users.

[riQQ]

See https://github.com/streetcomplete/StreetComplete/issues/3355#issuecomment-940203597 and https://github.com/streetcomplete/StreetComplete/issues/3355#issuecomment-940205653, i.e. westnordost doesn't want to do it himself but is willing to accept a contribution under the circumstances described.