initial fastlane structure

[IzzySoft]

As discussed by mail, here's a "starter package" for fastlane metadata (binaries not needed). Be welcome to my Fastlane Cheat Sheet for guidance on adjusting it to your wishes (e.g. adding a featureGraphic or per-release changelogs). Please note the limits mentioned there (e.g. shortdesc may not be longer than 80 chars, fulldesc 4000 chars, per-release changelogs 500 chars and plain ASCII, etc).

Once merged, I'll configure my updater to pull this along with future releases – so you are in full control of the content, and things like "old icons" should be things of the past :wink:

[umer0586]

@IzzySoft is this fastlane structure similar to what F-Droid uses?

[IzzySoft]

This is what F-Droid uses, yes. Both, F-Droid and my repo use the same structures (remember, my repo runs more or less the same fdroidserver, though with some minor adjustments). So even if you only merge it without modifications, F-Droid can use it directly – and when you create the next release tag, just let us know to update metadata there so we can remove the "hard-coded description & summary" so the one from here will replace it. Graphics (screenshots etc) should be picked up even without modification in F-Droid's YAML – currently, the "local metadata" would just override shortdesc & fulldesc, which is why we should remove them then.

[IzzySoft]

Thanks for merging! Please let us know then when the next release is planned, so metadata can be prepared at F-Droid accordingly (I just did that in my repo, so there it's already active and your changes are applied with each release).

[umer0586]

New release published

[IzzySoft]

And I just noticed this app is not (yet?) at F-Droid.org :see_no_evil: Still good you pinged me: added your new per-release-changelogs to my config so they are pulled along. Let me see how it works then:

$ iod repo get github.umer0586
github.umer0586: looking for 'https://api.github.com/repos/umer0586/SensorServer/releases'
github.umer0586: checking tag 'v3.2.0'
github.umer0586: lastRelNo set to 'v3.2.0', checking for files
github.umer0586: Upstream file date (2023-05-17 06:33) is newer than ours (2023-05-12 19:40).
github.umer0586: returning ['v3.2.0','https://github.com/umer0586/SensorServer/releases/download/v3.2.0/SensorServer-v3.2.0.apk',1684298013]
github.umer0586: 3.1.0/v3.2.0, https://github.com/umer0586/SensorServer/releases: https://github.com/umer0586/SensorServer/releases/download/v3.2.0/SensorServer-v3.2.0.apk
- Grabbing update for github.umer0586: OK
- Checking 'repo/github.umer0586_20.apk' for libraries and malware …
github.umer0586: check if repo contains FUNDING.yml
github.umer0586: looking for 'https://api.github.com/repos/umer0586/SensorServer/contents/.github'
github.umer0586: Github reports "Not Found" for https://api.github.com/repos/umer0586/SensorServer/contents/.github
github.umer0586: looking for 'https://api.github.com/repos/umer0586/SensorServer/contents/'
github.umer0586: no FUNDING.yml detected.
github.umer0586: calling 'getFastlaneMeta(github,[host:github.com,owner:umer0586,repo:SensorServer,path:/fastlane/metadata/android])'
github.umer0586: FastlaneFeatures shortdesc,fulldesc,icon,screenshotsJPG,changelogs
github.umer0586: looking for 'https://api.github.com/repos/umer0586/SensorServer/contents/fastlane%2Fmetadata%2Fandroid'
github.umer0586: looking for 'https://api.github.com/repos/umer0586/SensorServer/contents/fastlane%2Fmetadata%2Fandroid%2Fen-US'
github.umer0586: looking for 'https://api.github.com/repos/umer0586/SensorServer/contents/fastlane%2Fmetadata%2Fandroid%2Fen-US%2Fchangelogs'
github.umer0586: looking for 'https://api.github.com/repos/umer0586/SensorServer/contents/fastlane%2Fmetadata%2Fandroid%2Fen-US%2Fimages'
github.umer0586: looking for 'https://api.github.com/repos/umer0586/SensorServer/contents/fastlane%2Fmetadata%2Fandroid%2Fen-US%2Fimages%2FphoneScreenshots'
github.umer0586: checking locale 'en-US'
github.umer0586: updating 'metadata/github.umer0586/en-US/full_description.txt'
github.umer0586: updating 'metadata/github.umer0586/en-US/short_description.txt'
github.umer0586: updating 'metadata/github.umer0586/en-US/changelogs/20.txt' from '20.txt'
github.umer0586: updating 'repo/github.umer0586/en-US/icon.png'
github.umer0586: updating 'repo/github.umer0586/en-US/phoneScreenshots/01.jpg' as JPG
github.umer0586: updating 'repo/github.umer0586/en-US/phoneScreenshots/02.jpg' as JPG
github.umer0586: updating 'repo/github.umer0586/en-US/phoneScreenshots/03.jpg' as JPG
github.umer0586: updating 'repo/github.umer0586/en-US/phoneScreenshots/04.jpg' as JPG
github.umer0586: updating 'repo/github.umer0586/en-US/phoneScreenshots/05.jpg' as JPG
github.umer0586: updating 'repo/github.umer0586/en-US/phoneScreenshots/06.jpg' as JPG
github.umer0586: updating 'repo/github.umer0586/en-US/phoneScreenshots/07.jpg' as JPG
github.umer0586: cross-checking for obsolete screenshots
github.umer0586: screenshots in Fastlane: 01,02,03,04,05,06,07
github.umer0586: local screenshots checked: 01,02,03,04,05,06,07

Yay, that looks fantastico :smiley:

[umer0586]

Yes, this app is not available on F-Droid. Right now, I am learning the procedure for submitting the app there.

[IzzySoft]

Oh, that's easy enough. And when it comes to the metadata YAML file, I can provide the "upper half" by copying from my repo, should you want to go straight for the merge request – and as it's "pure Java", easily amend the rest:

metadata/github.umer0586.yml:

Categories:
  - Connectivity
License: GPL-3.0-only
AuthorName: Umer Farooq
SourceCode: https://github.com/umer0586/SensorServer
IssueTracker: https://github.com/umer0586/SensorServer/issues
Changelog: https://github.com/umer0586/SensorServer/releases

AutoName: Sensor Server

RepoType: git
Repo: https://github.com/umer0586/SensorServer

Builds:
  - versionName: 3.2.0
    versionCode: 20
    commit: v3.2.0
    subdir: app
    gradle:
      - yes

AutoUpdateMode: Version
UpdateCheckMode: Tags
CurrentVersion: 3.2.0
CurrentVersionCode: 20

So instead of opening an RFP and waiting for it to be manually processed, you could simply open a merge request – well, here it is. Let's see how it goes, pipelines are running now and first lights are turning green :smiley:

[IzzySoft]

Well, that was a straight hit – all engines are green :partying_face: And even with bonus points:

$ rbtest SensorServer-v3.2.0.apk github.umer0586_20.apk 
RB confirmed.

So it builds reproducible – fantastico! Let me update that MR. F-Droid then will ship the APK you build and sign, so cross-updates are easily possible.

[IzzySoft]

First issuebot report showed up: ALL ENGINES GO! :smiley: Marked for review. And as it's a reproducible build, no need to check if it "works as expected". But if you'd have a short test case for us, that could speed up things a bit (e.g. "start the app, do X, then curl <whatever>, you should see Z`). That way we can observe that it really only does what it's intended to. No mistrust here, but everyone expects us to make sure – after all, your app has a "dangerous combination" of permissions (location + internet = potential tracking).

[umer0586]

Thank you so much for creating a merge request there! 🌟 I was planning to issue a merge request with metadata/github.umer0586.sensorserver.yml since I recently updated the app ID to reflect uniqueness. However, I noticed that the metadata file (created by you) is named metadata/github.umer0586.yml. From now on, all my future releases will have the ID github.umer0586.sensorserver. Will there be any issues with this?

4c4a7600591c3b2af6ed060c8b96b672c30f7516

[umer0586]

Metadata file mentions following

Binaries: https://github.com/umer0586/SensorServer/releases/download/v%v/SensorServer-v%v.apk

So this means that APKs I publish here must follow SensorServer-v%v.apk naming pattern ?

[IzzySoft]

Will there be any issues with this?

Yes, that will be an issue: changing the applicationId means making it a different app altogether. Everybody who has the app installed needs to uninstall and reinstall – without getting a notice that they have to. No way to "simply update". So while the new applicationId looks better, it should be well decided if you want to go that step – and if yes, that must happen before that MR has been merged (and the MR must then be updated accordingly, with a new release being tagged etc. Without the renaming, whoever had your app installed before can simply update it from F-Droid.org, once it shows up there.

So if you want to do that, please put a note into the MR ASAP before someone reviews and merges.

So this means that APKs I publish here must follow SensorServer-v%v.apk naming pattern ?

Yes, that pattern must be kept by you then.

[umer0586]

Okay I'll then a put a note there once I publish a new release with new applicationId😢

[IzzySoft]

OK, marked the MR "Draft" so it doesn't get merged accidentally. Let me raise 3 questions/assumptions in this context, and a conclusion:

• the next release will have the new applicationId
• you want your app to stay in my repo after it appeared at F-Droid.org
• current users shall be forewarned

=> we should update the app description NOW to point to the upcoming change. Then when the new release is due, ideally let's have one APK with the current applicationId including a pop-up on next start pointing out the changes and requesting to switch. This APK could be prepared even in advance, and we just release it the same day the "new one" is released. My updater should "handle" that: it will grab the new APK and complain I have no metadata for it – at which point I grab the "switchy" one manually and publish both at the same day in parallel.

That way we can alleviate the problem. What do you say?

[umer0586]

the next release will have the new applicationId

Yes, v3.2.0 is the current version. Starting from v3.3.0 and onward, there will be a new applicationid

you want your app to stay in my repo after it appeared at F-Droid.org

I would consider keeping the app in your repo after it appeared at F-Droid.org. I always publish APKs on github's release section since there are people that directly install app by directly downloading APK from there. Github's analytics reports many views for release section.

current users shall be forewarned

I will refrain from providing advance notice to current users regarding the changes.

ideally let's have one APK with the current applicationId including a pop-up on next start pointing out the changes and requesting to switch. This APK could be prepared even in advance, and we just release it the same day the "new one" is released.

I appreciate the idea; however, I have already committed code changes since the release of v3.2.0.

[IzzySoft]

I appreciate the idea; however, I have already committed code changes since the release of v3.2.0.

And a manual build with the applicationId changed would not work? I'm no Android dev, so forgive my ignorance.

[umer0586]

https://apt.izzysoft.de/fdroid/index/apk/github.umer0586 still shows v3.1.0 as the latest version, whereas the actual latest version is v3.2.0, and I am planning to release v3.3.0 today with a new applicationId.

If I release v3.3.0 and include two binaries, v3.2.0 (with a pop-up) and v3.3.0 with the tag v3.3.0, would that cause a reproducible build failure at F-Droid?

[IzzySoft]

apt.izzysoft.de/fdroid/index/apk/github.umer0586 still shows v3.1.0 as the latest version

Huh? Following your link, I see v3.2.0, last updated 2023-05-17

If I release v3.3.0 and include two binaries, v3.2.0 (with a pop-up) and v3.3.0 with the tag v3.3.0, would that cause a reproducible build failure at F-Droid?

As v3.2.0 is already out, may I suggest to make that v3.2.1 and have the versionCode increased, so it's rolled out? As for reproducibility at F-Droid: I'll adjust the YAML once we're there. Just keep the naming pattern and either make it two different tags/releases, or keep the naming pattern for the new ID and give the popup-one a slightly different name (e.g. include something like "compat" or "deprec").

[umer0586]

Okay, l'll then publish v3.2.1 with two APKs

• SensorServer-v3.2.1.apk with new Id
• SensorServer-v3.2.1-deprecated.apk with old id having pop-up

[IzzySoft]

Sounds good, thanks! Please let me know when you'll release that, so I can take the appropriate measures here and at F-Droid.org.

[umer0586]

Okay

[umer0586]

@IzzySoft v3.2.1 published !

[IzzySoft]

Switch initiated in my repo, will now update the MR at F-Droid. You forgot to add the changelog for this version (fastlane/metadata/android/en-US/changelogs/21.txt), do you want to add that? Then I'd pull it for my repo (for F-Droid it won't be fetched as it's not at the tag, but that's no issue as there it's the first release anyway).

[umer0586]

Yes I forgot, but now I have added that

[IzzySoft]

Thanks! Updated here. So the switch will go live properly around 6 pm UTC then :smiley:

[IzzySoft]

No it no longer builds reproducibly:

Binary files /tmp/tmp.Qx8fNAUlCY/a/res/3m.xml and /tmp/tmp.Qx8fNAUlCY/b/res/3m.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/3n.xml and /tmp/tmp.Qx8fNAUlCY/b/res/3n.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/8E.xml and /tmp/tmp.Qx8fNAUlCY/b/res/8E.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/8_.xml and /tmp/tmp.Qx8fNAUlCY/b/res/8_.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/Ah.xml and /tmp/tmp.Qx8fNAUlCY/b/res/Ah.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/C5.xml and /tmp/tmp.Qx8fNAUlCY/b/res/C5.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/cR1.xml and /tmp/tmp.Qx8fNAUlCY/b/res/cR1.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/Cv.xml and /tmp/tmp.Qx8fNAUlCY/b/res/Cv.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/DV.xml and /tmp/tmp.Qx8fNAUlCY/b/res/DV.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/ew.xml and /tmp/tmp.Qx8fNAUlCY/b/res/ew.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/FZ.xml and /tmp/tmp.Qx8fNAUlCY/b/res/FZ.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/hE.xml and /tmp/tmp.Qx8fNAUlCY/b/res/hE.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/J7.xml and /tmp/tmp.Qx8fNAUlCY/b/res/J7.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/j9.xml and /tmp/tmp.Qx8fNAUlCY/b/res/j9.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/jj.xml and /tmp/tmp.Qx8fNAUlCY/b/res/jj.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/Jk.xml and /tmp/tmp.Qx8fNAUlCY/b/res/Jk.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/lA.xml and /tmp/tmp.Qx8fNAUlCY/b/res/lA.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/m8.xml and /tmp/tmp.Qx8fNAUlCY/b/res/m8.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/mX.xml and /tmp/tmp.Qx8fNAUlCY/b/res/mX.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/OR.xml and /tmp/tmp.Qx8fNAUlCY/b/res/OR.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/rE.xml and /tmp/tmp.Qx8fNAUlCY/b/res/rE.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/u9.xml and /tmp/tmp.Qx8fNAUlCY/b/res/u9.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/w2.xml and /tmp/tmp.Qx8fNAUlCY/b/res/w2.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/WF.xml and /tmp/tmp.Qx8fNAUlCY/b/res/WF.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/WG.xml and /tmp/tmp.Qx8fNAUlCY/b/res/WG.xml differ
Only in /tmp/tmp.Qx8fNAUlCY/a/res: wh1.xml
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/wh.xml and /tmp/tmp.Qx8fNAUlCY/b/res/wh.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/xc.xml and /tmp/tmp.Qx8fNAUlCY/b/res/xc.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/res/xd.xml and /tmp/tmp.Qx8fNAUlCY/b/res/xd.xml differ
Binary files /tmp/tmp.Qx8fNAUlCY/a/resources.arsc and /tmp/tmp.Qx8fNAUlCY/b/resources.arsc differ

Checking with our howto: Did you really build from the tag, or did you use a different commit? In the latter case: which (and for the future: don't!)?

PS: Can we move further discussions on this to the corresponding MR – i.e. can you join us there?

[TotallyAvailable]

@IzzySoft I do have to wonder (even more with your recent "debug cert elimination" list, which I absolutely support just like pushing for as few "risky" permissions as possible) how many reproducible F-Droid apps actually carry a debug cert underneath. ('MobileInpact' apparently as well)

Shouldn't those be "eliminated" as well ?

After deciding not to post a lengthy "risk assessment", shouldn't reproducible builds either only be based on a production cert or be clearly marked as debug/test. (anti-feature/regardless of the cert being public or not)