release on fdroid?

[jakenvac]

original issue comment:

as the title suggests, I think an fdroid release would be great.

Edit: I made this issue on mobile, so sorry for the terseness.

F-Droid is a pretty standard release channel for open source apps, especially so for things like matrix clients & other privacy focused tools.

I think the F-Droid version would need the google crash reporting removing and an alternative avenue for notifications. Something like ntfy or another unified push solution would work well.

I understand that this app is still in early access, so this issue is more for visibility, tracking discussion etc.

[ouchadam]

it's definitely planned for the future, before it can be provided via fdroid we'll need different mechanisms for receiving notifications such as background polling and/or unified push

ultimately I would like to have 1 variant of the app that can be published on both the google play and fdroid stores, with FCM (firebase cloud messaging) disabled if the device doesn't have google play services available

[jakenvac]

If you want to publish to the official f-droid repository, I don't think they'll allow a single app with non-foss components. I could be wrong though.

Unified push is definitely a great solution for devices without GAPPS.

[ouchadam]

ah you're right https://f-droid.org/en/docs/Inclusion_Policy/ my use case is explicitly called out (google play services and crashlytics) :sweat_smile:

the release build is also non deterministic due to the R8 optimizations

it's a shame but with that in mind I'll put fdroid on the backburner, having to maintain multiple variants and being unable to track crashes in a single place is a deal breaker for me

Prebuilt APKs and the source will always be available on github and I'll still plan to add support for non FCM based update mechanisms

[frainz-de]

I this case, you might consider adding the app to IzzyOnDroid's F-Droid repository. Its inclusion policy is less strict and it takes the APKs directly from Github

[ouchadam]

this looks like a great compromise, thanks for the link!

Reopening this issue to track meeting IzzyOnDroid's inclusion policy

[IzzySoft]

Asking my library scanner, it responds with:

Offending libs:
---------------
* Crashlytics (/com/crashlytics): NonFreeDep,Tracking
* Firebase Data Transport (/com/google/android/datatransport): NonFreeNet
* Google Mobile Services (/com/google/android/gms): NonFreeDep
* Firebase (/com/google/firebase): NonFreeNet,NonFreeDep

4 offenders.

I have the same reservations here which were voiced above. This is a messenger one should trust with quite personal details, so there should be no big-tech trackers (here: Crashlytics, GMS, Firebase) aboard.

Also already mentioned was @UnifiedPush – which, if you need Firebase for FCM, is an excellent alternative and gives users the choice to pick the notification backend they prefer (for PlayStore, you could even include their FCM distributor). If you need other features from that Firebase set, there's appwrite and Supabase.

For more details and hints, here's my F-Droid snippet – and here are some details on Push Clients/Managers :smiley:

If you agree taking steps on this path towards F/LOSS and privacy, I'd accept your app as soon as Crashlytics is removed. It will then still be listed with the other 2 anti-features (NonFreeNet & NonFreeDep), but they'd be hopefully removable as your development goes on. Once everything is "cleaned", I can also help you moving on to F-Droid.org (disclosure: I'm one of the maintainers there – and hundreds of apps have been going this path, using my repo as stepping stone, which is one of its main purposes :wink:)

Please let me know if that would work for you, and I'll gladly help you getting started.

[ouchadam]

now that small-talk supports unified push #82 , I'll look to create a foss fork with a patch to remove the private dependencies

[IzzySoft]

While on it: Seeing #121 and if you consider honoring it, maybe take preparations that it doesn't get in the way – which mainly means being careful not to drag in proprietary dependencies, but also considering avoidance of the NonFreeNet anti-feature. For example by using a wrapper class so you can replace the underlying translation service; there seem to be some FOSS services (like LibreTranslate) out there as well you could then "swap" using build flavors.

[ouchadam]

thanks for the heads up! I ended up doing something similar for firebase push messaging and crashlytics by creating no-op replacements https://github.com/ouchadam/small-talk/pull/105

it should now be possible to build a foss variant with ./gradlew assembleRelease -Punsigned -Pfoss although I haven't checked with any 3rd party scanners yet

[IzzySoft]

If there's an APK, I can scan it. Or you can use my scanner yourself, it's FOSS. Details and instructions can be found in my article Identify modules in apps :wink:

Oh, don't wonder if the scanner still reports your "stubbed" replacements. It will still see the same paths in the smali. As for Firebase Push: why not use UnifiedPush, and include their Firebase distributor for your Playstore variant while leaving it out for the "plain foss" variant? That way F-Droid users could simply use it with their preferred distributor (several are available at F-Droid, see Push Clients/Manager).

[ouchadam]

The amount of code needed for the firebase integration is quite minimal and shouldn't require too much maintaince, delegating a core functionality is a little risky IMO and we can see there's some growing pains with Element's integration

The no-op delegates use intermediate package names to help avoid false positives (assuming they don't get optimised away!)

I'll start adding unsigned FOSS apks to the generated github releases, unfortunately due to R8/code shrinker the builds won't be reproducible (I'm hoping they add the ability for a seed in the future 🤞) and the use of jetpack compose reallyyyy benefits from the R8 optimisations for runtime performance

[IzzySoft]

Thanks for considering, @ouchadam – though I must say that I'm unsure whom an unsigned APK would help: an end-user cannot easily install it, and if it were reproducible it could not be double-signed (by you and by F-Droid) without being signed by you first :wink: Also, the way you're choosing by going "direct Firebase" means FOSS users won't have the notifications. AFAIK, integrating UP is not (much?) more work – but would serve both.

[ouchadam]

ah I was under the impression the fdroid repositories acted as key signers and required unsigned builds

for notifications there might be some confusion, SmallTalk supports UP providers 😄 https://github.com/ouchadam/small-talk/pull/82

[IzzySoft]

I was under the impression the fdroid repositories acted as key signers and required unsigned builds

Oopsa… OK, clarification: F-Droid would clone your repo and build the APK. If set to reproducible, your signed APK would then be pulled, the signature removed, and the two unsigned APKs would be compared with diffoscope. If that gives a thumbs-up, your signed APK would be picked, F-Droid adds its own signature on top, and the resulting APK would be published in the F-Droid repo – this way allowing for cross-updates (e.g. if you'd run your own repo, or if you push some "urgent update", the APK installed from F-Droid can be updated with your APK as the signature matches. Without "reproducible build" users would need to uninstall first.

No worries, you could start with your app being signed by F-Droid only and switch to reproducible builds whenever you're ready. In that case, a signed APK from you would still serve end-users well (at least until your app is listed at F-Droid) – especially in cases of trouble with PlayStore :wink:

for notifications there might be some confusion, SmallTalk supports UP providers

Ah, great – then indeed I was being confused :rofl: All fine then, thanks!

[ouchadam]

thanks for the detailed explanation! for the next steps what's needed from my side? (for the simpler non deterministic builds)

I have automatic releases on mondays & thursdays which increment the versions, tag and build/publish apks (when there are changes to release)

eg https://github.com/ouchadam/small-talk/pull/127 https://github.com/ouchadam/small-talk/releases

[IzzySoft]

I have automatic releases on mondays & thursdays which increment the versions, tag and build/publish apks (when there are changes to release)

I was hesitant to pick them as they are declared pre-release. If you say they are ready-to-be-used I can go ahead and add your app to my repo… wow, wait, I can't:

Offending libs:
---------------
* Crashlytics (/com/crashlytics): NonFreeDep,Tracking
* Firebase Data Transport (/com/google/android/datatransport): NonFreeNet
* Google Mobile Services (/com/google/android/gms): NonFreeDep
* Firebase (/com/google/firebase): NonFreeNet,NonFreeDep
* Firebase Installations (/com/google/firebase/installations): NonFreeNet

5 offenders.

Those marked NonFreeDep are show-stoppers for F-Droid – and this amount (more than 4) is very much border-line even for mine (plus something sensitive like a personal messenger should according to the rules of my repo have no trackers inside). We'd need a build (flavor) coming without them. Which most likely means: without all 5 of them, as e.g. Firebase Installations would drag in Firebase + GMS as dependencies IIRC (not sure what Firebase Data Transport would drag in). So to answer your question

for the next steps what's needed from my side?

Have a build flavor coming without those dependencies (or remove them altogether). I can only guess what you're needing Firebase for: seeing the c2dm permission, one of the reasons most likely is "cloud notifications". These could e.g. be replaced by @unifiedpush (for PlayStore builds, you then could include their Firebase distributor, and your Google users wouldn't notice any difference – while leaving the distributor out of the FOSS flavor and F-Droid users freely can pick their preferred distributor.

If there are other things you need from Firebase, you might wish to take a look at appwrite or Supabase which are intended as FOSS replacements.

That leaves Crashlytics. Take a look here for some FOSS alternatives to that.

Next step would then be setting up metadata. I can help with that if we got the above dependencies at least reduced a bit (so I can take your app into my repo), as then I'd have to setup metadata anyway and could offer the results via PR.

[ouchadam]

I've added an unsigned foss build to the latest release for scanning 08/09/2022-V1 | .apk

FOSS builds are a gradle parameter (-Pfoss) rather than build flavor

[IzzySoft]

If you could sign it, I could add it to my repo immediately. Scanner says:

Offending libs:
---------------
* Firebase Data Transport (/com/google/android/datatransport): NonFreeNet
* Google Mobile Services (/com/google/android/gms): NonFreeDep
* Firebase (/com/google/firebase): NonFreeNet,NonFreeDep
* Firebase Installations (/com/google/firebase/installations): NonFreeNet

4 offenders.

So it's not yet meeting F-Droid's inclusion criteria – but if I close both eyes it matches mine. Let's target dropping the others with future releases so you can apply for F-Droid.org then. With your app in my repo, the library section for the corresponding release will tell you when that goal has been reached :wink:

[ouchadam]

looks like an annotation import ended up including firebase and its transitive dependencies into the resulting apk :facepalm: is fixed with https://github.com/ouchadam/small-talk/pull/136, checking for firebase references in the apktool output looks clean :crossed_fingers:

Here's a signed release https://github.com/ouchadam/small-talk/releases/tag/09%2F10%2F2022-V1-EA

[IzzySoft]

Yay, that looks fine now – great, thanks! Give it one more hour to show up here with the next sync around 6 pm UTC. I've also opened a PR adding fastlane structures to your repo here – you'll need that when applying for inclusion with F-Droid.org. And if you could add some screenshots to it (see the cheat-sheet linked from the PR) and let me know, that'd be wonderful: I'll then enable Fastlane with everything in it to be pulled along with each future release.

Speaking of which: Your versionNames are a bit strange, and the tag names even more. Let's see how my updater will handle that, but a recommendation would be using semantic versioning and have matching tag names (e.g. version 1.2.3 at tag v1.2.3). But most important: please keep the string foss-signed as part of the FOSS APK so my updater can find and pull the correct one (I've told it to ignore all others).

[ouchadam]

for mobile apps I'm a (fairly recent~) convert for https://calver.org/ where most users simply want the latest version (me included :smile: )

please keep the string foss-signed as part of the FOSS APK

will do! :+1:

[IzzySoft]

most users simply want the latest version

That's probably true for most users. However, updates should work – which they currently don't, due to slashes (/) in the versionName:

app.dapk.st: 08/09/2022-V1/22/09/2022-V1, https://github.com/ouchadam/small-talk/releases: https://github.com/ouchadam/small-talk/releases/download/22/09/2022-V1/foss-signed-22.09.2022-V1.apk
app.dapk.st_22/09/2022-V1.apk: No such file or directory
- Grabbing update for app.dapk.st: Fail (no data from'https://github.com/ouchadam/small-talk/releases/download/22/09/2022-V1/foss-signed-22.09.2022-V1.apk')

Will check if I can fix that with my updater in case it shows up somewhere else, too (in 6 years and with thousands of apps, this is the first case with that issue) – OK, done, works now (after I also enabled pre-releases for your app).

[mi5o]

Just for clarification: Does the latest SmallTalk fit F-Droid inclusion criteria? It seems it is neither available in F-Droid repo (https://search.f-droid.org/?q=app.dapk.st), nor requested for inclusion at GitLab (https://gitlab.com/fdroid/rfp).

[IzzySoft]

How can we tell if it was not submitted? As a raw guide: from what my library scanner says about their APK, it looks good – the app is listed in my repo as you know.

PS: Why not open an RFP and find out?

[mi5o]

How can we tell if it was not submitted?

AFAIK, all submissions to F-Droid go via GitLab issues to their RfP project. Hence, if there is no issue, either open or closed, then most likely nobody has requested it yet, I believe. You can look for yourself: https://gitlab.com/fdroid/rfp/-/issues.

As a raw guide: from what my library scanner says about their APK, it looks good – the app is listed in my repo as you know.

Yes, I know it is there. (And my instance of Neo Store keeps an eye on it, but somehow it happens to skip @ouchadam's releases, so for the time being, I rely on Github releases.) But I know that inclusion criteria for your repo are more relaxed, than those of F-Droid, so while a successful inclusion is definitely a good sign, it is not convincing. Yet I feel not competent enough to evaluate the inclusion questionnaire, probably only @ouchadam has sufficiently intimate knowledge of the app. As he already evaluated, at least once in the past.

PS: Why not open an RFP and find out?

In one word: it would be impolite in my eyes. They kindly ask to check the app first, whether it fulfils their inclusion criteria, whether it has already been submitted or even accepted, and only ask for packaging then. I am ready to ask for inclusion, it needs not to be done by the author. Hoping that if the build fails, @ouchadam will find the necessary adjustments worth his effort, particularly after it is already in your repo. But at least, I should be somehow confident the app seems to fulfil those criteria, before submitting it. From @ouchadam's comments on various places in Github and chat, coming from different times, I was unable to learn, what he believes is the current state of affairs.

[IzzySoft]

AFAIK, all submissions to F-Droid go via GitLab issues to their RfP project.

Yes, either that or it was added directly via a merge request at fdroiddata. But my question was how we can tell whether it meets the inclusion criteria if it was NOT submitted – i.e. Issuebot has not scanned it. So to answer your original question: you can find out by submitting it there.

I feel not competent enough to evaluate the inclusion questionnaire,

Then leave that check-mark unticket – as many others did before you (and even I do before the bot reported: I didn't write the app so I need a guide, too). We'll tick the boxes ourselves then based on bot reports and progress.

They kindly ask to check the app first, …

Yes, we do. But we do not expect that everybody would be able to answer all the questions ahead. I'd say in half the cases initial runs fail and some work is needed. Consider those questions a check-list of tasks being performed during the process: if you know in advance one item has to fail (e.g. because your app heavily relies on some proprietary library and cannot do without) it of course makes no sense wasting your and our time. But if you don't know: well, some RFPs have to be closed unsolved.

Oh: and I say "we" as, full disclosure, I'm one of the maintainers there :wink: So your politeness and taking our criteria serious like that sent a warm shiver down my spine, thanks a lot!

And TL;DR: Feel encouraged to submit the app. Remember one of the criteria is the author being INFORMED and NOT OPPOSED? It doesn't say it must be the author opening the issue. And as you can see in the second post above, @ouchadam is even supporting it, quote: "it's definitely planned for the future". I see the "but" in that post as well, so let's wait for a go (are you ready, @ouchadam?)

[ouchadam]

from my side I think the app is fully compliant, I was originally hesitant about the work required to create a separate FOSS build but that turned out to be easier than expected :smile:

I'll have a look at raising the rfp

[erebion]

I'll have a look at raising the rfp

Have you done that already? What happened?

[ouchadam]

I've gone down the self hosting route https://ouchadam.github.io/fdroid-repository/repo/ which is powered by this repository https://github.com/ouchadam/fdroid-repository which simply uploads the latest release

I'll look to release to the main fdroid repository once the app reaches a non beta status