FOSS build flavour for F-Droid

[Bubu]

I just tried upgrading the app on f-droid, but I noticed that the recent releases (2.0.2 and 2.1) introduced crashlytics resp. google play services, which are closed source libraries.

Could you provide a gradle build flavour that doesn't use these libraries so the app can be updated on F-Droid again?

[andrewnenakhov]

Can you contact me over email? info@xabber.com, got some questions for you

[Bubu]

email sent

[cl0ne]

Any news on upgrade on f-droid?

[andrewnenakhov]

We'll attend to it once we'll fix some lags in modified interface.

[Human]

@Bubu Other than a foss build flavor that doesn't include Google Play Services or Crashlytics, are there any other requirements?

[Bubu]

@Human No, I don't think so.

[Human]

@Bubu Where is the check done? On the resulting APK? You just run 'zipinfo' (or equivalent) on it to look for something like com.crashlytics and reject the build if it's found?

[Bubu]

@Human There are some checks performed on the build.gradle files before building the app. See mostly here: https://gitlab.com/fdroid/fdroidserver/blob/master/fdroidserver/scanner.py#L65 (blacklist) and here: https://gitlab.com/fdroid/fdroidserver/blob/master/fdroidserver/scanner.py#L101 (whitelist of maven repos)

[Human]

@Bubu My understanding is that a build flavor can customize how the project is built and what ends up in the APK, but it doesn't change what's in the sources. If the sources are scanned before even trying a build, I think it'll always fail the check here because several build flavors do use Google Play Services and Crashlytics, requiring the google repo and the crashlytics aar to be in the gradle files. So, even if a foss build flavor didn't end up putting them into the APK, the sources still reference them.

Because each build flavor does get its own src subdirectory, in theory, the foss flavor could have its own version of whatever java sources used google or Crashlytics blobs, but this would be akin to maintaining a fork or a special branch of the main project, since it'd always be behind the main build flavors and require periodic updating. And if your checks don't ignore proprietary blobs found in other src subdirectories, this would fail anyway.

[Bubu]

@Human The scanner is pretty dumb but it does recognize the flavorCompile directives and only produces errors on those matching the flavour we are building.

The rest that ends up being in build.gradle but is not actually required for that flavour is mostly removed via a prebuild=sed -i -e '/fabric/d' build.gradle command in the fdroid metadata file or something similar to this.

This is done for a lot of apps that use a foss/non-foss flavour, you'll find plenty of examples, i.e. here: https://gitlab.com/fdroid/fdroiddata/blob/master/metadata/com.gh4a.txt#L91

[Human]

@Bubu I can do some more investigation, but the offending libraries are referenced via classpath and not compile. There appears to be no flavor-specific classpath prefix.

[Bubu]

Right, that is one of the cases where the fdroid recipe would sed out that line. It would just be worth testing that this fdroid/foss flavour indeed keeps working when these lines are removed/commented out.

[Human]

@Bubu Do I also need to remove the Facebook library? com.facebook.android:facebook-android-sdk:4.31.0 I didn't find any facebook checks in the code you linked to earlier.

[Bubu]

@Human That library seems to be open source: https://github.com/facebook/facebook-android-sdk

So there is no problem in including that into an fdroid build.

[Human]

@Bubu Thanks. There's another snag I'm working through. There are several keys and secrets for 3rd-party login mechanisms stubbed out in ./xabber/src/dev/res/values/social_keys.xml. For dev builds, the app will build and run, but anything relying on these keys will fail. For prod builds, it is my understanding that core project members have a copy of the fleshed-out file locally. Since the foss build flavor is supposed to be "prod minus closed-source stuff" the missing keys and secrets are a problem.

You might want to ask a core project member for permission to maintain a copy of that key file in your build environment. Otherwise, the f-droid version will lack the login API features available to the Play Store version.

[Bubu]

There are only two possible solutions to that problem right now:

• Disable those features in the Foss build.
• have this values injected in the fdroid build process. These values would then live in the fdroiddata repo being public for anyone who would look close enough. What apis are we talking about?

On March 23, 2018 5:32:52 PM GMT+01:00, Bob Igo notifications@github.com wrote:

@Bubu Thanks. There's another snag I'm working through. There are several keys and secrets for 3rd-party login mechanisms stubbed out in ./xabber/src/dev/res/values/social_keys.xml. For dev builds, the app will build and run, but anything relying on these keys will fail. For prod builds, it is my understanding that core project members have a copy of the fleshed-out file locally. Since the foss build flavor is supposed to be "prod minus closed-source stuff" the missing keys and secrets are a problem.

You might want to ask a core project member for permission to maintain a copy of that key file in your build environment. Otherwise, the f-droid version will lack the login API features available to the Play Store version.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/redsolution/xabber-android/issues/770#issuecomment-375724346

[Human]

@Bubu All of the keys and secrets are used in BaseLoginActivity.java The APIs are 3rd-party authentication to create / access a "Xabber account." Facebook, Google, and Twitter are supported in the code base. (The Github auth is stubbed out and nonfunctional at the moment.)

I have not personally tried any of the features requiring a Xabber account, and I haven't explored how deeply the integration goes. I don't know yet if it's something that could be turned on or off with a flag, or if the notion of a "Xabber account" is pervasive throughout the app. I'd have to investigate further.

In parallel with my investigation, would you find out if the core dev(s) would be ok with f-droid having access to the keys and secrets in some capacity? If they were to say yes, that might be simpler and faster than conditionally hiding the relevant UI.

[Human]

@Bubu It's actually pretty simple to just bypass the splash screen that asks you how you want to log in, so if there's no way to get the 3rd-party credentials, it can be disabled at the root.

[andrewnenakhov]

There are only two possible solutions to that problem right now:

• Disable those features in the Foss build.

@Bubu we planned to disable social auth features in F-droid build for exactly that reason. If keys issued to us by FB&GOOG&Twitter are used in the wild, it can be used by anyone under our credentials, which is bad.

[andrewnenakhov]

I would also ask to wait a bit more before building for F-droid, we're in the process of fixing some important bugs that plague current version, it'd be better to move to F-droid once we're finished with this phase.

[Human]

@andrewnenakhov If you let me know when you'd be comfortable having an F-droid build, I can merge and continue the work. At the moment, I have a handle on all the required issues except for a clean way to avoid including crashlytics code. @Bubu The solution for that may have to be pre-editing the relevant lines out.

[gerroon]

Hi

Is there any update on this build work? I realize this is some work, I am just trying to see when we can have the latest version in F-droid

thanks

[andrewnenakhov]

@gerroon sorry for delays. We're currently very overloaded making exciting new features like very nice and user-friendly media sending and also combating a significant amount of technical debt that surfaced in android version of Xabber. So knowing of some severe issues that resurfaced while introducing new functionality we have 2 options: to build from an already obsolete version of Xabber or to add some great features, fix everything, make it stable and then make a build for f-droid. We choose the latter.

[gerroon]

@andrewnenakhov

That is fair enough. I realized it must be a lot of work already. Take your time, Xabber is one of the most loaded Xmpp client already.

thanks

[IzzySoft]

May I ask what is the status here? F-Droid users certainly would be very excited for the exciting features as well – if they one day were available at F-Droid :wink:

Reference: issue 1184 at fdroiddata

[IzzySoft]

Well, no answer after half a year doesn't exactly raise hope. Have you given up, and we should mark the app as "no more updates" at F-Droid, or even move it to archive? It's way behind there, the last available version being v2.0.1 from 5/2017, i.e. being more than 2 years old. @andrewnenakhov ? You've had a lot of releases since. Any chance we get this tackled?

[Human]

Maybe forking is the right solution?

[IzzySoft]

@Human of course you can fork, fix and provide a PR. I can't (well, I can fork, but then I'm stuck – not being an Android developer and all that). To clarify why I chimed in: I'm one of the F-Droid maintainers, walking through our issue queue and trying to figure what is stuck and where. I do that about 2 or 3 times a year for house-keeping.

So no offense meant, but may I be so bold and ask the status again – given that I cannot fix the code myself? :smiley:

[andrewnenakhov]

@IzzySoft @Human we're still committed to doing this. Just really busy :(

we initially planned to do lots of improvements and roll it out all together with our server and ios version and web version but we ran into some problems with app performance and stability, fixing this took a lot of time. Maybe we'll not try to roll out everything at once and will just release a copy of what we have on Google Play. Hopefully, next week.

[IzzySoft]

Hi @andrewnenakhov – that sounds great, thanks! Looking forward to it. Feel free to ping me if I should update the issue on our end to get some packager to take a look (when there is a fitting tag, that is). "Next week" is more than I dared hoping for!

[Str4tocaster]

Hello @IzzySoft I am currently working on version of Xabber Android for F-Droid. I have a question about com.google.android.gms:play-services-safetynet dependency. Have F-Droid restriction on this dependency or not? We use recaptcha from this packet to signup process in our cloud service.

[Bubu]

@Str4tocaster I'm pretty sure nothing under com.google.android.gms:* is free software. So it cannot be included in F-Droid.

[IzzySoft]

Apart from that, ReCaptcha is a nightmare for users, not only privacy-wise. Whenever I encounter it, I close the tab/app/whatever straight away. I'm tired of marking buses, store-fronts, water hydrants … over and over, only to finally be allowed to check a box (last time I gave in, I gave up at round 10). I'd strongly recommend dropping that.

[andrewnenakhov]

@IzzySoft well it depends on how it set up and how many attempts occur with similar parameters. In our case for most registered users it doesn't appear even once, acting invisibly. However, some users did report that on some devices it just fails to load, making registration impossible. IN this case we suggest registering with our (or any other server) website to obtain an account.

Also, for now, we decided to do just the same for F-droid build: when user presses on 'sign up' button we'll simply open a system browser pointing to registration url.

[IzzySoft]

Meaning, the fdroid flavor would come without ReCaptcha (and without any other GMS/proprietary dependencies), right? Then it should be fine (pending approval from @Bubu – I'm not familiar with the "build stuff").

[andrewnenakhov]

Yes, without ReCaptcha

[Str4tocaster]

@Bubu can you check branch feature/fdroid for compliance F-Droid requirements? I tried to remove all proprietary dependencies. The build variant for F-Droid is openProdRelease.

[Bubu]

Looks good from a short glance.

[Str4tocaster]

@Bubu What our next steps to release Xabber on F-Droid?

[andrewnenakhov]

@Bubu what should we do next? Just wait?

[Bubu]

I have no time to do anything here, sorry.

[IzzySoft]

@rudloff could you maybe pickup where Bubu left?

[Rudloff]

I tried to build a7da670802882b2e1ae867e6ade2fd4bf15267fd with the open variant and it seems to work as expected. Now, we would need a stable tag that includes this variant.

[IzzySoft]

Thanks Pierre!

@andrewnenakhov your turn again, answering the "what should we do next": ^^ 😃

[Str4tocaster]

@Rudloff thanks for testing! We will prepare release build and make a stable tag soon.

[licaon-kter]

Should this be a new app: com.xabber.android like on Google Play?

Or the old org.xabber.androiddev ?

/LE Any news on https://github.com/redsolution/xabber-android/issues/857 ? I still see connections even before trying to sign up or in... :(

[Str4tocaster]

@Rudloff @IzzySoft The stable tag https://github.com/redsolution/xabber-android/releases/tag/v2.6.6 is here. We are ready to return at F-Droid!

[licaon-kter]

@Str4tocaster and some answers?

[Str4tocaster]

@licaon-kter com.xabber.android

[licaon-kter]

/close this

Ref: https://gitlab.com/fdroid/fdroiddata/commit/d782601729e9c2ee4e9210dfc3b4fe4dfd5bcc76