search

lichess-org / lichobile

lichess.org mobile application
https://lichess.org/mobile
GNU General Public License v3.0
2.04k stars 318 forks source link

Remove Firebase Analytics #1121

Closed IzzySoft closed 2 years ago

IzzySoft commented 4 years ago

corresponding to #590 – your switch to FCM accidentally dragged that in. AFAIK it's a simple toggle with the wrong default, incidentally done that way by Google.

veloce commented 4 years ago

Please elaborate. I made sure to disable analytics in firebase console, so where do you see analytics?

IzzySoft commented 4 years ago

My library scanner reported that, see here (scroll down and click on "libraries detected"). Triggered by /com/google/firebase/analytics/connector being present in smali.

veloce commented 4 years ago

I think it's good now: https://github.com/veloce/lichobile/commit/045d6eea1d8e548383177bb66d90f6b89c03d375

IzzySoft commented 4 years ago

Thanks! Looks good from what I can tell about the code (I'm not an Android dev). "For sure" I'll be able to tell as soon as the next APK is available and has passed my scanner (maybe you'll tag v7.0.1?). Will close this issue then when confirmed.

veloce commented 4 years ago

Just published: https://github.com/veloce/lichobile/releases/tag/v7.0.1

IzzySoft commented 4 years ago

Thanks! Updater already grabbed it. And scanner still complains about Firebase Analytics. Let me look at the smali from v7.0.1… That's strange, it looks identical to that of v7.0.0 – diff doesn't find a difference :cry:

veloce commented 4 years ago

Too bad. I don't see what I can do about it now. What is strange is that the gradle settings had really an effect: I could see in the logs that firebase-analytics could not be initialised.

IzzySoft commented 4 years ago

Yeah, the times of "don't be evil" are long gone – Google likes to sneak things in everywhere. I've got a bunch of other apps in my repo using FCM which don't have this issue, so there must be some way to get that out. Not being an Android dev myself I unfortunately cannot help figure. If it helps you I could check for some "example apps" also hosted on Github, maybe you see some "trick" in their code?

veloce commented 4 years ago

It could help, yes.

IzzySoft commented 4 years ago

OK, then let me list some examples (going by the CDM permission and Firebase being detected; no guarantee they're not still on GCM):

While looking for these examples I've noted that most apps using FCM also show Firebase Analytics, probably for the same reason as here. I've skipped apps last updated 2018 or earlier, as they are much more likely to use GCM.

seniorm0ment commented 4 years ago

On the F-Droid version of Lichess, (possibly through a 3rd party repo?) it shows Lichess requires Google Services, Firebase Cloud Messaging and Firebase Analytics, and are listed as anti-features. What I don't understand, is why any of these things are needed in the app anyways? Lichess claims to be a Libre, no-ads, open source chess server. Why does this stuff need to be implemented within the app that basically goes against those ideals?

Can we please just remove (not "disable", keyword being remove) all these anti-features, and just keep it as Lichess. It would be great to see the app follow the ideals of the platform. I highly encourage this decision.

veloce commented 4 years ago

They are here because of push notifications. If you don't want to use push notification on lichess app and want to have this libraries removed you can build your own version of the app using this repo.

seniorm0ment commented 4 years ago

They are here because of push notifications. If you don't want to use push notification on lichess app and want to have this libraries removed you can build your own version of the app using this repo.

How do most other apps achieve push notifications curiously without using such? Is it just to stay alive in the background without it killing it? Because I note most apps do generally have a stay alive feature that throws the app into notification area (which you can minimize). Otherwise I don't really understand why Lichess needs these for push notifications, but thousands of other FOSS apps get away with notifications without them?

IzzySoft commented 4 years ago

@grravity they are probably using other alternatives, like MQTT, XMPP or SSE (the latter is e.g. used by Tutanota & Mastodon), or web sockets (Telegram, Signal).

@veloce while I generally agree, not everyone is able to "self-compile". For a "naked build" (without push at all for now), could you establish a build flavor? Then we might be able to have F-Droid build that.

seniorm0ment commented 4 years ago

they are probably using other alternatives, like MQTT, XMPP or SSE (the latter is e.g. used by Tutanota & Mastodon), or web sockets (Telegram, Signal).

Not too familiar with any of this, I am slightly familiar with XMPP as far as communications goes, but yes this does make sense as to me as to how/why. I'm more confused as to why Lichess isn't using these originally and instead using Google services.

while I generally agree, not everyone is able to "self-compile". For a "naked build" (without push at all for now), could you establish a build flavor? Then we might be able to have F-Droid build that.

I believe most do use two different flavors. Google services for their Play Store release, and then have an F-Droid release not using Google Services. I think this is the best way to go if it's possible in this case, and kinda what I was originally hoping for and expected upon originally going to download the app. Was confused when saw Google services was required for the F-Droid release. And yes, even if I could self-compile without them (and still be stuck without push which sucks), it doesn't account for everyone else who doesn't know how or can't as well as you stated, forcing them to use Google services to play Lichess through the app (or don't play at all)

Kinda sucks, Lichess is amazing especially it's libre aspect, but then we have to have Google services within the F-Droid version of the app, which ruins it. Hoping some changes with the F-Droid version will be considered here. :)

IzzySoft commented 4 years ago

I'm more confused as to why Lichess isn't using these originally and instead using Google services.

Wild guess: what's the first thing you think of when someone mentions Push? What's the most wide-spread push implementation? And which one is the easiest to implement? We're not born as privacy advocates. It's something some of us reach at some point. So my guess is that @veloce simply wasn't aware of the implication when he added FCM – and possibly would approach that differently today. But once you're "locked in", replacing isn't as easy as the original integration.

Was confused when saw Google services was required for the F-Droid release.

There is no separate F-Droid release as of now. It's the very same APK you find in Play, which @veloce thankfully attached to the releases here at Github so I can fetch it into my repo. As you can see above, he does his best to keep it privacy-friendly. So hopefully, one day we'll have a variant that comes completely without those proprietary components. When that will be, and how it will be done – we may contribute our ideas, but it's done when it's done and how the dev(s) decide(s).

Thanks @veloce for bearing with us tin-foils!

seniorm0ment commented 4 years ago

@IzzySoft Yes but the site and app seem very clear about being libre and privacy friendly, it seems to be only common sense not to use Google Services, or just some lazy implementation

ian-tedesco commented 4 years ago

So, I've been lurking through #409 and this (https://gitlab.com/fdroid/rfp/issues/94), and it seems that the only way we get an F-droid version is if Firebase analytics are removed, and push notifications are changed for a FLOSS alternative on the regular APK uploaded to G Play store, or if someone builds another one specially for F-droid without these.

I think the first way would be the better, since there are a lot of people who are not privacy advocates, or are even aware of the critical stalking condition taken by Silicon Valley who download the app from the most common application store and would have their data being stolen without them ever knowing it. This would also make easier the idea of having an F-droid version since it would be handled by the same person.

Or am I missing something?

By the way, you can check if your application has trackers in it by using Exodus. https://reports.exodus-privacy.eu.org/en/

seniorm0ment commented 4 years ago

@5a384507-18ce-417c-bb55-d4dfcc8883fe This, I also think just removing them completely would be easier than dealing with two flavors (FOSS and NonFOSS) and It is also what I would suggest.

Also Exodus Privacy works good, but I find ClassySharkExodus to work a bit better and finds things Exodus Privacy doesn't, but it is a little more "overwhelming" to look at for some.

IzzySoft commented 4 years ago

I find ClassySharkExodus to work a bit better and finds things Exodus Privacy doesn't

That's because it also includes my library definitions, in addition to Exodus', and probably some more (as mine get added to Exodus as well occasionally) :smile:

but it is a little more "overwhelming" to look at for some

Oh yes. That's why I had recommended that "intermediate page" with the short listing, which ClassyShark luckily included. The 3 of us (the author of ClassyShark, Exodus and me) often exchange ideas.

ian-tedesco commented 4 years ago

By the way, while this is being worked out (and I assume it's going to take a while since this problem is being dealt with for at least two years), you can use the progressive web app which makes things a bit nicer than playing from your browser and allows you to block scripts if you have uMatrix/uBlock Origin (although all of Lichess' scripts are necessary and there isn't any third party tracker or similar thing).

gary-host-laptop commented 4 years ago

Any updates on this? I noticed you uploaded a version of Lichobile to your repository @IzzySoft, what about that? Is the exact same version that is on Google Store?

IzzySoft commented 4 years ago

It's the one you find attached to releases/ here. If that's the same that goes to Play, I cannot tell.

veloce commented 4 years ago

It's the same.

seniorm0ment commented 4 years ago

I assume this means this (removing non-foss depdencies) still isn't even being worked on currently? Unless I'm understanding wrong?

veloce commented 4 years ago

I tried (cf. discussion above) and failed. Feel free to try and submit a PR.

seniorm0ment commented 4 years ago

Rip, alright I don't have the knowledge to do this so I unfortunately cannot provide help myself. Hope to see someone come around and give it a go.

gilbsgilbs commented 4 years ago

Just adding my two cents here, I'd love to see Lichess app free from proprietary dependencies, but this may actually be harder than one could think. Just creating a build flavor without push services wouldn't be sufficient because capacitor itself has transitive dependencies to firebase-messaging and play-services-location. I guess they need to address this first before Lichess can do anything.

Mageek627 commented 3 years ago

I have a quick-and-dirty fix that should work. I forked the Capacitor repository (https://github.com/Mageek627/capacitor) in order to remove everything related to Firebase and Google Services. After cloning that you do:

cd capacitor/core
npm run build
cd capacitor/cli
npm run build

Then in your project:

npm install path/to/capacitor/core
npm install path/to/capacitor/android
npm install -D path/to/capacitor/cli

I built the apk of the Ionic blank app with Capacitor and analyzed it in Android Studio and I don't find anything Google-related. I'm not sure that it doesn't break anything critical with Capacitor though.

Whenever I'll have the time, I'll try to integrate this into a version of lichobile without Firebase (and therefore with no notification), so that we can at least have a working version soon. Of course if someone else feels like doing that, don't wait for me!

Mageek627 commented 3 years ago

@LongJohn-Silver @grravity Thank you for the emojis, I think adding some on the Capacitor issue that I just created would be even better to get a long-term solution.

gary-host-laptop commented 3 years ago

@Mageek627 Could you link me to it, please?

Mageek627 commented 3 years ago

@LongJohn-Silver sure https://github.com/ionic-team/capacitor/issues/4176

Mageek627 commented 3 years ago

According to https://github.com/ionic-team/capacitor/issues/4176#issuecomment-775501496 Capacitor 3 will be without Firebase by default, so it should be an easy transition. If someone makes a libre fork/branch, it would be (hopefully) temporary until Capacitor 3 becomes stable.

gary-host-laptop commented 3 years ago

Why is it completely necessary to wait until it reaches stable? Nightly versions work good sometimes.

Mageek627 commented 3 years ago

@LongJohn-Silver I meant that the decision to include beta code in the main version of Lichess would rest ultimately with Vincent Velociter. In the meantime for sure we can make a parallel version with the beta code, but it would hopefully not stay a long-term thing, because the goal would be that the main version is libre.

veloce commented 3 years ago

For sure I don't have the will nor the time to maintain a parallel version with a forked capacitor. I you want to build such a no-google version, you can do it in a fork.

Now I've already done everything possible (from what you can get in firebase documentation) to remove the tracking, and leave only the push notification service. As you can see here:

If I find the time some day, what I could do (or I could accept a PR that does) is a gradle build flavor that would remove push notifications.

seniorm0ment commented 3 years ago

@Mageek627 I unfortunately do not have much knowledge to contribute, I was just curious however I noticed you forked Lichobile.

Were you planning on working on and releasing a de-googled fork or submitting a PR? Or were you just playing around?

Mageek627 commented 3 years ago

@grravity I don't have any concrete plans yet. I'm not working on it currently either. But I'll let you know if I do.

gilbsgilbs commented 3 years ago

Now that Capacitor 3.0 is released, it should be slightly easier to make a build variant that doesn't depend on Firebase (because they removed the PushNotification thing from core). I can see at least those steps:

Then we have to find a way to remove the dependency to the @capacitor/push-notifications plugin when building the OSS flavor. This could be done with a simple sed -i '#@capacitor/push-notifications#d' package.json, but then push.ts wouldn't be very happy about the missing dependency. So either:

As far as I know, unlike with Android/iOS native development, there's no way officially supported way to use different dependencies or code for different build types in capacitor. I'd be curious to hear should somebody knows a better way to handle this. Maybe I lack some experience with this framework.

If either solution fits to Lichobile maintainers, I'm ok spending some time working on this.

veloce commented 3 years ago

I will upgrade to capacitor 3.0. About the rest, I'm ok with slight changes that don't involve extra maintenance cost over the long term. But we'll see that after.

Baikalcadillac commented 2 years ago

Any updates on this? Capacitor 3.0 was released some time ago.

EvanCarroll commented 2 years ago

Not sure if this issue is dead, but I would also really like to be able to download lichess on F-droid. Even if there was notifications didn't work. There is a lot of talk about a build being easier after capacitor 3 is adopted. But it seems it is?

https://github.com/lichess-org/lichobile/blob/3c6572b0e77dffec008311c47ea010c9f12022eb/package.json#L57

What's the hold up now on making an f-droid build?

@gilbsgilbs did you any traction on your proposed changes? https://github.com/lichess-org/lichobile/issues/1121#issuecomment-849844594

gilbsgilbs commented 2 years ago

I didn't know the Capacitor 3 upgrade had been done, therefore I did not make any progress since then. But this is great news.

I reckon my previous comment still holds: after removing @capacitor/push-notifications dependency from package.json, firebase-messaging dependency from app/build.gradle, replacing push.ts with a stub, and performing some 🔞 tricks to make the build work on my machine, I was able to play my first game on a firebase-free APK.

Glancing through push.ts a bit more carefully though, I fear there might be some side-effects I did not anticipate earlier to stubbing the implementation completely. Some "in-game specific" listeners are declared and it seems that lila itself sends those events to Firebase. I'm not completely sure of the consequences if we shut them entirely and why the websocket isn't sufficient. Maybe for correspondence games? Or if the app goes in background and the system decides to kill the connection? Anyhow, we need to understand the exact consequences.

Powersource commented 2 years ago

Does this mean lichess will be available on f-droid? (I was directed here by that question)

gilbsgilbs commented 2 years ago

@Powersource F-Droid metadata PR still needs to be made, but I don't think there's anything blocking inclusion anymore as far as lichobile in concerned.

IzzySoft commented 2 years ago

Until then it's available in my repo still. Once the new APK was fetched I can check for remaining stoppers; apart from Firebase Analytics, the previous versions also include GMS & FCM which would also be show-stoppers.

As for metadata: I could provide what's set up with my repo (Fastlane structures as F-Droid expects) as a starter-package if wanted.

gilbsgilbs commented 2 years ago

I started to make the recipe and realized that there's at least two issues.

Apart from that, I was able to build the free variant successfully.

IzzySoft commented 2 years ago

Missing fastlane metadata.

I can repeat my above offer, @gilbsgilbs:

I could provide what's set up with my repo (Fastlane structures as F-Droid expects) as a starter-package if wanted.

You just need to say you want them. And yes, in your repo is much preferred – F-Droid itself won't accept graphic files any more (the metadata repo was growing too much and became hard to handle). You can then adjust what I send of course, but you will at least have the structures in place and not to guess what goes where :wink:

gilbsgilbs commented 2 years ago

Sure @IzzySoft, please do share what you have, don't ask 😄.

Regarding upstream inclusion of fastlane metadata, as it doesn't depend on me but rather on what the maintainers of lichobile want, I'll completely ignore this issue for know and see what happens.

Also, it seems the bintray repository was deprecated. I opened #2184 to remove it which sounds like the cleanest solution.

IzzySoft commented 2 years ago

There you go :smiley: