Publish Android App to F-Droid

[x2764tech]

Would it be possible to get the Android App published on F-Droid? I believe they are happy to have pre-release versions available alongside stable versions.

[geeksville]

good idea - will do soonish

[geeksville]

This issue has been mentioned on Meshtastic. There might be relevant details there:

https://meshtastic.discourse.group/t/first-post-as-a-test/13/1

[geeksville]

Hmm I did some investigation. Alas, the rules for submitting to fdroid doesn't allow crashlytics. Which is super useful for a young project like this (we get about one crash report every day for our 80ish users, and that report usually has enough info for us to fix the bug). They mention ACRA as being an acceptable alternative - alas, the workqueue is super full right now. So I'm punting this for something after 1.0, but if someone wants to submit a PR to change to ACRA instead of crashlytics (assuming it has similar features) I'd be fine with that.

In the meantime (for those that can't access google play), we provide apks via github releases.

[geeksville]

ok - closing until someone else wants to do this (analytics are too useful to ban, instead we let user choose to opt in)

[fossdd]

Yepp, your app needs Crashlytics, Google Play Services, firebase and mapbox. These are a no go for us. It was opened a RFP for this app, but with these things we are unable to include that.

Like you sad in https://github.com/meshtastic/Meshtastic-Android/issues/184#issuecomment-705297424, using ACRA would work. The F-Droid client also uses ACRA.

[geeksville]

@fossdd yep understood. alas, until someone contributes PRs to make those changes we probably won't be doing that (because other tasks are higher priority). Keep being awesome.

(We do push APKs to our github releases though in case this helps other searchers)

[IzzySoft]

We do push APKs to our github releases though

That sounds like a signal to me. But unfortunately, that's even too much for my repo – which allows for "minor transgressions" (with the idea of them getting fixed so the app can probably progress to "F-Droid proper"), but also has a limit (3-4 are acceptable, but not more). Here come 6 of them, as my library scanner reports:

Offending libs:
---------------
* Crashlytics (/com/crashlytics): Tracking
* Firebase Data Transport (/com/google/android/datatransport): NonFreeNet
* Google Mobile Services (/com/google/android/gms): NonFreeDep
* Firebase (/com/google/firebase): NonFreeNet,NonFreeDep
* Firebase Analytics (/com/google/firebase/analytics): Tracking
* Mapbox SDK Telemetry component (/com/mapbox/android/telemetry): Tracking

6 offenders.

[geeksville]

understood, alas, moving away from mapbox (and installing/hosting ATAK) is very low on our list of priorities. I think f-droid is a great idea, but until someone sends in a PR to fix these things it probably won't be happening.

[IzzySoft]

Also understood. My point was rather that there's a gray between black and white. If it were just Mapbox, even F-Droid would probably accept it (just adding the Tracking anti-feature). If there were a flavor/APK without Crashlytics & Firebase, I could already accept it into my repo (despite of GMS still being there, though that would be a show-stopper for F-Droid itself).

So should in the interim you be able to reduce those "offenders" to 3 or below, give me a ping and I check again. Should one day they be reduced to zero, it can be moved on to "F-Droid proper". More than 200 apps already went that path (starting in my repo and then "moving over" when all issues were solved).

No pressure, just a "friendly pointer" :wink:

[geeksville]

btw @IzzySoft thanks for mentioning GMS, I hadn't noticed my common utilities baselibe (geeksville-androlib) still ahd a reference to that. I wasn't using it - I just removed it from the build.gradle and the next APK I release (later today here in Taiwan time) won't reference it.

https://github.com/meshtastic/geeksville-androidlib/commit/49ac3e94f9302c3cb684aef9bc7b70a1ebb0d989

[IzzySoft]

Ah! That's very good news! If GMS is gone, Firebase should disappear as well (AFAIK Firebase depends on GMS). Please give me another ping once the APK is available, so I can check it. Thanks a lot!

[goyalyashpal]

just commenting here for explicitly saving. seems there are some very good things that i can learn from this issue and ongoing discussions.

found the rfp report via https://gitlab.com/fdroid/rfp/-/issues?scope=all&utf8=%E2%9C%93&state=closed&search=signal

[SkewedZeppelin]

Here is a starting point if anyone wants to create a fork free of those proprietary dependencies. It compiles, but I am unsure how to include the Android-variant of the device firmware. BLE device detection might require the user manually grant the app location permission. It also lacks all maps functionality. https://gist.github.com/SkewedZeppelin/b82fb2969139a525af6df7d4ed555882

[Mhowser]

any further updates to this?

[IzzySoft]

Still has Crashlytics enabled, so no way. An option would be using build flavors and having one without the proprietary dependencies for F-Droid. As I guess the majority of users will still install from Google (or rather Amazon if I read this correctly), it wouldn't mean a hit on crash reports – and both sides should be satisfied.

[gjedeer]

I've done some work on it, but there's still a long road to an fdroid-friendly build.

Here's my code -> https://github.com/gjedeer/Meshtastic-Android/commit/10135486ba3327b1032e713bcb8c8509a324c665

My main contribution here is the build.gradle fdroid flavor. ./gradlew assemblePlay will build the original version while ./gradlew assembleFdroid will build the smaller version.

The kotlin code is mostly from @SkewedZeppelin except I did not remove mapbox. It's not sustainable, it's basically a fork in a subdirectory, the code would probably need to have some refactoring so that the fdroid flavor doesn't need to retain a copy of most of the code (in app/src/fdroid and app/src/play).

And more importantly, the code is highly dependent on https://github.com/meshtastic/geeksville-androidlib , which pulls its own copy of play services and firebase analytics.

If someone wants to take a look at the apk -> https://f.gdr.name/meshtastic-fdroid-debug-20220109.apk

[gjedeer]

The library was surprisingly easy -> https://github.com/gjedeer/geeksville-androidlib/commit/55d00742164f79fd82bf57bdccc707d72dac24d8 - and this is something I think can be merged, as it just replaces most of the code with stubs, and contains no copies of actual working code.

apk build with the f-droid flavor of the library -> https://f.gdr.name/meshtastic-fdroid-debug-20220109-v2.apk

@IzzySoft what program are you using to generate these "offending libs" tables?

[gjedeer]

Also, location sharing is not going to work, as only Play Services location provider is implemented. But that's minor and quite easy to add if someone is motivated.

[IzzySoft]

what program are you using to generate these "offending libs" tables?

My own library scanner. Can be found here (scanapk.php, needs some files from the lib/ directory as well (libsmali.txt and libinfo.txt containing the library definitions) where you also can find the description). scanapk.php is the "simple variant" working stand-alone (just requiring those two definition files), so you don't need the complete setup with the entire framework – just those 3 files, plus ApkTool of course.

Quick setup:

• create an empty dir to work in, place scanapk.php there
• create a subdir lib and place the library definitions as well as Apktool there (the Jar and the shell script will be needed)
• adjust the first line in scanapk.php to read $apktool = './lib/apktool'; (pointing to the shell script)
• the second should read $libdir = './lib'; then

Now you should be able to process APKs in that directory, like ./scanapk.php my.apk (with the APK in the same directory). Adjust to your liking :wink:

[gjedeer]

Then: yeah!

Offending libs:
---------------
* Mapbox SDK Telemetry component (/com/mapbox/android/telemetry): Tracking

1 offenders.

Thanks @IzzySoft

[SkewedZeppelin]

You can put in the manifest: <meta-data android:name="com.mapbox.EnableEvents" android:value="false" />

https://github.com/mapbox/mapbox-events-android/pull/77

[gjedeer]

@SkewedZeppelin nice! Added in the f-droid flavor https://github.com/gjedeer/Meshtastic-Android/commit/77295f7995f818092dc24b7043292705ace0a1e8

[IzzySoft]

@gjedeer library definitions are updated about once a month. The script rarely changes (works now). Just in case you want to make sure to be up-to-date when using it. At F-Droid we use a variant that simply downloads the library definitions anew at each run :smile:

[Mhowser]

Is there any possibility that we could use OpenStreetMap as a drop-in replacement for mapbox?

Edit: I found this library that could be used https://github.com/osmdroid/osmdroid

[SkewedZeppelin]

drop-in

no

[gjedeer]

There's a lot of work done for separating the fdroid-incompatible code in this branch: https://github.com/abduldblogger/Meshtastic-Android/

Not ready for a PR yet because the latest code is not merged, but if someone wants to test or review, that would be very helpful.

[andrekir]

@IzzySoft can I ask for an updated review and possibility of Meshtastic app being included in your fdroid repo? we've since removed all offending libs except for Firebase Analytics / Crashlytics.

[IzzySoft]

I've just checked the APK from the latest pre-release (2.0.12). If that's the one you refer to I must disappoint (otherwise please tell me where to find the one you mean):

Offending libs:
---------------
* Crashlytics (/com/crashlytics): NonFreeDep,Tracking
* Google Ads (/com/google/ads): Ads,NonFreeDep
* Firebase Data Transport (/com/google/android/datatransport): NonFreeNet
* Google Maps API (/com/google/android/gms/maps): NonFreeDep,NonFreeNet
* Google Mobile Services (/com/google/android/gms): NonFreeDep
* Google Play Core (/com/google/android/play/core): NonFreeNet,NonFreeDep
* Firebase (/com/google/firebase): NonFreeNet,NonFreeDep
* Firebase Analytics (/com/google/firebase/analytics): NonFreeDep,Tracking
* Firebase Installations (/com/google/firebase/installations): NonFreeNet

9 offenders.

So compared to the last check, there are now even more "offenders" present. And why do you insist on those 2 proprietary Google trackers? Does none of the FOSS ones fit your needs? Also instead of GMaps, won't OSM do?

[andrekir]

thanks for the info. surprising, but good to know.

we certainly don't run ads and already use OSM instead of GMaps. the only gms library present is com.google.gms:google-services (per Firebase Crashlytics reqs) so I wonder if that's where all those google libs are coming from.

I can prepare and link here a couple debug APKs to investigate further, is it possible to get scan results from debug APKs?

[IzzySoft]

I wonder if that's where all those google libs are coming from.

Maybe a dependency of a dependency? gradle :app:dependencies could help you figure out.

is it possible to get scan results from debug APKs?

Sure. And you can even do that yourself if you wish (so you needn't wait for me), see my comment from last year above :wink:

[gjedeer]

@andrekir please read the thread above, me and other people have done research which non-free dependencies are used in which components. But TLDR, some of this stuff is in "androidlib" to which I've sent a PR, and the elephant in the room is mapbox.

[andrekir]

hello @gjedeer! I am aware, see my comment on your PR. androidlib has since been archived and Mapbox replaced with OpenStreetMap thanks to yours and others' great feedback in this thread.

working on the last steps to load project/gradle.build "classpath" and app/gradle.build plugins only for the play store build.

should have something to share and test soon.

[andrekir]

@IzzySoft can you please check pre-release 2.0.13 fdroidRelease APK?

[IzzySoft]

Good job! That's acceptable for my repo now, just not for F-Droid.org:

Offending libs:
---------------
* Google Play Core (/com/google/android/play/core): NonFreeNet,NonFreeDep

1 offenders.

So shall I take it in – or rather leave the prereleases alone to be distributed via your repo? Speaking of which: maybe you can create a dedicated badge for that? Using the F-Droid.org badge to point somewhere else than F-Droid.org is a bit misleading. A quick hack would be replacing the F-Droid logo by the Meshtastic one, and alter the text to "Get it on our F-Droid repo". Should your app one day be at F-Droid, you'd need that anyway (unless you plan to shut down your repo then).

[andrekir]

almost there. I'll remove that last lib in the next release, then I want to submit a request for packaging in the mainline F-Droid repository. As for the badge, you bring up a valid point I didn't consider. I'll take care of it. thank you!

[IzzySoft]

Thanks a lot, @andrekir! Please give me another ping here once the last offender should be gone – so I can remove the AntiFeatures and confirm "readiness for mainline".

[andrekir]

@IzzySoft can you please check the latest pre-release 2.0.14 fdroidRelease APK?

[IzzySoft]

Congrats! No offenders anymore. So for auto-updates outside of Google Play, be welcome to pick a badge to put next to the existing one and send people here (link will become valid tomorrow around 7 pm UTC). I've picked the version you just linked, but set my updater to ignore pre-releases; if you want it to pick up those as well just let me know. Otherwise, starting with the next "proper release" (i.e. one not marked pre-release) it will pick only those. Oh, and it will check daily.

Once you've reached F-Droid.org we can see how to proceed: having the app stay at my repo, having it only pick up betas while F-Droid gets the stable ones, or remove it from my repo again altogether :wink:

Thanks for all the effort you've put into it!

PS: next step fastlane structures? If you want to I can send you what I've set up for my repo, so you'd have a "starter package". It's loosely based on the app's PlayStore listing.

[geeksville]

btw Andre - I'm sorry to be mostly retired from coding/meshtastic but OMG so great that you became the new badass android leader. keep rockin it!

On Fri, Jan 27, 2023 at 1:16 PM Izzy @.***> wrote:

Congrats! No offenders anymore. So for auto-updates outside of Google Play, be welcome to pick a badge https://gitlab.com/IzzyOnDroid/repo/#if-my-app-is-available-here-do-you-have-a-badge-i-can-use-to-link-to-it to put next to the existing one and send people here https://apt.izzysoft.de/fdroid/index/apk/com.geeksville.mesh (link will become valid tomorrow around 7 pm UTC). I've picked the version you just linked, but set my updater to ignore pre-releases; if you want it to pick up those as well just let me know. Otherwise, starting with the next "proper release" (i.e. one not marked pre-release) it will pick only those. Oh, and it will check daily.

Once you've reached F-Droid.org we can see how to proceed: having the app stay at my repo, having it only pick up betas while F-Droid gets the stable ones, or remove it from my repo again altogether 😉

Thanks for all the effort you've put into it!

PS: next step fastlane structures https://gitlab.com/-/snippets/1895688? If you want to I can send you what I've set up for my repo, so you'd have a "starter package". It's loosely based on the app's PlayStore listing.

— Reply to this email directly, view it on GitHub https://github.com/meshtastic/Meshtastic-Android/issues/6#issuecomment-1407082889, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABXB2PAFS7SVDYLZC4WUN3WUQ3L3ANCNFSM4LKOK6RQ . You are receiving this because you modified the open/close state.Message ID: @.***>

[andrekir]

thank you for your kind words, @geeksville! :heart:

@IzzySoft excellent, thank you! if we can keep metadata in a single place it would be ideal, we have Tripe-T structure in https://github.com/meshtastic/Meshtastic-Android/tree/master/app/src/main/play would that work for yours and mainline repos? I opened a new request for packaging issue here: https://gitlab.com/fdroid/rfp/-/issues/2340

[IzzySoft]

Triple-T will work for F-Droid.org (my updater does not support it). And yes, I saw the RFP, looks good so far! May I recommend adding a FUNDING.yml? Then your OC (and whatever you might add later) would be picked up automatically – by my repo as well btw.

[licaon-kter]

any chance to remove mil.nga:mgrs from the fdroid flavor? It's not FOSS: https://github.com/ngageoint/mgrs-android/blob/2.2.2/mgrs/build.gradle#L159

ref: https://github.com/meshtastic/Meshtastic-Android/pull/473/files

[andrekir]

@licaon-kter com.google.android.gms was addressed back in https://github.com/meshtastic/Meshtastic-Android/issues/6#issuecomment-1386180753, and was since excluded from the lib import:

https://github.com/meshtastic/Meshtastic-Android/blob/4f9011fe57d6fcfc85f0561a217a32ae09ef0a6d/app/build.gradle#L219

there are no offending libs in our fdroid release since we were added to IzzySoft's repo.

[licaon-kter]

and was since excluded from the lib

That's not how it works. :)

The lib depends on proprietary libs, it NEEDs proprietary libs to be built, hence any artifact of it (.aar from Maven) is not FOSS.

If there's a way to remove gms lib and rebuild the library, yes, we can use that, we can rebuild it before building your app.

If not...

Anyway, Izzy repo has different rules for the APKs it pulls.

[andrekir]

@licaon-kter understood. I replaced the library with only the base API, please check if this change solves the issue.

[licaon-kter]

this recipe builds

License: GPL-3.0-only
SourceCode: https://github.com/meshtastic/Meshtastic-Android
IssueTracker: https://github.com/meshtastic/Meshtastic-Android/issues

RepoType: git
Repo: https://github.com/meshtastic/Meshtastic-Android.git

Builds:
  - versionName: 2.2.24
    versionCode: 30224
    commit: 4ba67beb532dc760d066d1b76e219f07b1fc4e48
    subdir: app
    submodules: true
    sudo:
      - apt-get update
      - apt-get install -y openjdk-17-jdk-headless
      - update-java-alternatives -a
    gradle:
      - fdroid
    prebuild:
      - sed -i -e '/(useCrashlytics)/,+3d' -e '/firebase/d' -e '/gms/d' ../build.gradle build.gradle

AutoUpdateMode: None
UpdateCheckMode: Tags

can anyone test the APK? com.geeksville.mesh_30224.apk.ZIP (remove .ZIP from name)

[andrekir]

can anyone test the APK? com.geeksville.mesh_30224.apk.ZIP (remove .ZIP from name)

it works! can we use reproducible builds?

Binaries: https://github.com/meshtastic/Meshtastic-Android/releases/download/%v/fdroidRelease-%v.apk

thanks!

[licaon-kter]

to be able to test that you need to provide an APK built from https://github.com/meshtastic/Meshtastic-Android/commit/4ba67beb532dc760d066d1b76e219f07b1fc4e48

[andrekir]

sounds good! I will ping you once a new release is ready.

[licaon-kter]

there's no need for a release if you're not ready, you can build an APK, sign it and attach it here