Open Ramaddan opened 7 years ago
Hello, is there any progress on this issue? This small fix would be very nice, as a righteous app such as this should also be free as in freedom. :-)
@ahmedre Any updates on this issue?
any progress her?
i'll replace Crashlytics with Firebase Crashlytics which is open source. that should fix the problem and allow submission of the app there in sha' Allah.
السٌَلامُ عَلَيْكُمْ وَ رَحْمَةُ اللهِ وَ بَرَكاتُهْ any progress on this :) ?
Wa alaikum assalam.
Crashlytics is already replaced with Firebase Crashlytics. So, in theory, the app should be compatible with F-Droid now.
in that case i urge the dev to submit the app at : https://gitlab.com/fdroid/rfp
Assalaamu alaykum
Yes please. It would be great to have it there.
On January 19, 2021 12:32:09 PM GMT+02:00, bingoxo notifications@github.com wrote:
in that case i urge the dev to submit the app at : https://gitlab.com/fdroid/rfp
-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/quran/quran_android/issues/755#issuecomment-762752982
Wa alaikum salam.
Sorry, apparently, Firebase Crashlytics is also not acceptable.
https://f-droid.org/docs/Inclusion_Policy/:
We cannot build apps using proprietary tracking/analytic dependencies like Crashlytics and Firebase.
perhaps @licaon-kter can shed some light on alternatives ?
wa3laikum alsalam,
Firebase Crashlytics is not hard to toggle on/off - this isn't a big issue to do. The bigger problem here is that the app has some code (mostly resource files) that come from a separate (private) repo. This is likely against the terms of F-Droid. Building without this (optional) repo means the version in F-droid would only have 2 reciters, no extra page types, etc. If this restriction of building without this library is ok, I can make Firebase Crashlytics optional.
Alternative to Firebase?
It's not opensource, but ACRA or Bugsnag are.
Private repos? Yeah, that's a showstopper
wa3laikum alsalam,
Firebase Crashlytics is not hard to toggle on/off - this isn't a big issue to do. The bigger problem here is that the app has some code (mostly resource files) that come from a separate (private) repo. This is likely against the terms of F-Droid. Building without this (optional) repo means the version in F-droid would only have 2 reciters, no extra page types, etc. If this restriction of building without this library is ok, I can make Firebase Crashlytics optional.
are you the maintainer of the private source repo ? how can we make it open ? aren't most resources available openly on the internet ? i read some old post about people forking it to add ads and make money of it , is that what's holding all this back ?
yes, I am the maintainer of the private repo. It used to be part of this repo.
The problem isn't just a personally annoying aspect of "adding ads and making money", there are other problems:
I had 2 choices - completely close the source moving forward (that's what happened with the iOS app, for example), or keep things open but close out the resources to discourage people from inflicting this harm. hence I went with the second approach - the app you can build here is 100% functional, it just limits the set of available (out of the box) recitations to 3 recitations and the available page types to 1 page type if I only compile with the open source code.
@ahmedre Salam alaykum! As for servers (for both audible content and ayahs), you might want to try this public CDN: https://alquran.cloud/cdn - there are API instructions there, and there are no license restrictions on the content. If we use a public CDN, at least for the FOSS version, then shouldn't this issue be gone for good? For contact details, you might want to implement a system that hashes the APK, sends the hash to a server operated by the app's team (could be a free-tier Heroku instance and a Node.js script), then only files a ticket if the hash of the APK matches an official release. (Or you could possibly remove the "submit ticket" option from the open-source version) And finally, for ads ... this is one tough nut to crack. Perhaps you can file Play Store copyright takedown requests on the offending apps?
I'd love to hear what you think about these options.
wa3laikum alsalam, the problem is the bandwidth cost of transferring the mp3s - api is extremely lightweight and only accessed very rarely, so can run off a $5/mo virtual server (even with several folds the traffic). the hashing solution could work for mp3s for example but would have to figure out a way for it to work in a backward compatible manner.
my question is would a Quran for Android version that lives in F-droid with only a handful of Qaris be better than nothing being there? if so we can make this happen pretty easily (just have to fix making Crashlytics optional for compilation). this would be the fastest way to make it happen. walsalam 3alaikum.
@ahmedre That CDN does exactly that - it has MP3 files available for public use. You can integrate URLs from that website in your app (since it's license-restriction-free) instead of MP3 files from your server - you'd essentially slash MP3 hosting costs to zero. You can redirect the current links on your own server to links that point to that other server - this way, backward compatibility is preserved. If it comes down to either nothing or an open-source version with just a handful of qaris, then the latter is definitely better, but let's hope we can figure out a more functional alternative! :-D
they have 5 different Qari types (and all gapped not gapless) whereas the app today ships with a lot more. same thing about shipping with a limited audio subset if so would apply here.
@ahmedre How about this one? https://mp3quran.net Sample URL: https://server11.mp3quran.net/yasser/024.mp3 They also have a BitTorrent service: https://www.mp3quran.net/ar/quran-download I couldn't find a license for the content, but the website is up for the sole purpose of service Quran MP3 files, so I can't see how they would object to this being used in an open-source app.
Otherwise, we could go ahead with a very limited selection, or to implement moar verification. But verification is a cat-and-mouse game, and there's no way it would be backward-compatible (older versions would lose access, possibly compounding the issue)
Can @bingoxo or whomever has access to the F-Droid ticket explain the situation to them and ask them if we can get an exception?
For this reason, source-built applications are the preferred method for the main F-Droid repository, although occasionally for technical or historical reasons, exceptions are made to this policy.
https://f-droid.org/en/docs/Signing_Process/
Yes. In all but a very few cases (for technical or historical reasons), we build all applications directly from the source code.
https://f-droid.org/en/docs/FAQ_-_App_Developers/#will-my-app-be-built-from-source
just for clarity, our content is also free (as far as we know, we take anything down whenever we get copyright notices about it), and we're hosting anyway and don't plan on changing that in sha' Allah whether on F-Droid or not.
i think the point is people taking the app and abusing it (disrespecting the Qur'an) on one hand, while also using our resources and bandwidth on the other hand. realistically, the financial cost to us from these apps is likely negligible. i don't want to make this easy for people and as we've seen time and time again on both iOS and Android, people being able to clone and re-ship the project as is with all features and things makes the barrier to doing this extremely low, thereby increasing how often this happens.
i'd vote for a limited set of qaris - but yeah if whoever has access to F-Droid can give an exception for just having a set of URLs not being part of the open source build, that'd make things easier.
also, for people who can't / don't want to install Google Play, we always have apks on our Github releases page.
is the apk on release page the 1:1 copy of the one in playstore ? from a privacy standpoint is there a benefit to get it from there instead of play store ?
as for the exception on f-droid , maybe @licaon-kter can again give more insight
Exceptions? No we can't... the app could just download the assets on first start or ask the user to provide them locally?
@bingoxo To me, the primary useful feature of F-droid is updates, and given the app is relatively mature, I don't need automatic updates, and can just check the releases every once in a while.
If you have a trust issue, either
Here is another suggestion: Would you be happier having an apk on releases without Crashlytics for example (yes, I think the apk is the same on playstore)? (Not sure if @ahmedre likes this option)
@licaon-kter
Exceptions? No we can't... the app could just download the assets on first start or ask the user to provide them locally?
While we can possibly try to ask for an exception (they very clearly said "technical reasons"), the solution that you provided (make the open source version load the assets from local storage) is better, especially since the required assets can be compressed and uploaded to any static file host, or even a peer-to-peer file sharing website. However, a multitude of online-Quran-MP3 websites exist, why not just leverage these as an alternative?
@benomaire Is FOSS all about trust and nothing more?
Yes. Many people put their trust in FOSS. So it mainly depends upon trust and community development.
ٱلسَّلَامُ عَلَيْكُمْ وَرَحْمَةُ ٱللَّٰهِ وَبَرَكَاتُهُ I would vouch for a trimmed down version on FDroid, that would atleast be a start.
I think a good interim solution for users could be the IzzyOnDroid repo, the APKs are taken directly from GitHub releases, and it can be added to F-Droid so users get automatic updates and don't have to use Google Play: https://apt.izzysoft.de/fdroid/index/apk/com.quran.labs.androidquran
@IzzySoft can you add the other apks (in release page , there are 4 versions for every tag) to your repo please ?
@abdulocracy
it can be added to F-Droid so users get automatic updates
Note that they won't be able to update from Izzy's (Github version) to F-Droid version since they are signed by different keys.
@abdulocracy
it can be added to F-Droid so users get automatic updates
Note that they won't be able to update from Izzy's (Github version) to F-Droid version since they are signed by different keys.
there is no f-droid version at the time , he probably meant add izzy's repo to fdroid
Was just a fyi :)
And I can only add 1 APK per version. Oh, wait: You're not talking about different ABI builds, but completely different apps? Yes, that would be possible – provided they use each their own unique applicationId
(which IIRC they do). But then I'd need the short and the full description for each, plus the entire set of metadata (I never figured what the difference is between the 4). OK, they're all on Play so I can check there. Let's see:
Naksh:
Offending libs:
---------------
* Crashlytics (/com/crashlytics): NonFreeDep,Tracking
* Firebase Data Transport (/com/google/android/datatransport): NonFreeNet
* Google Mobile Services (/com/google/android/gms): NonFreeDep
* Firebase (/com/google/firebase): NonFreeNet,NonFreeDep
* Firebase Analytics (/com/google/firebase/analytics): Tracking
5 offenders.
Nope. Only if you can slim down that list. Too many NonFree & proprietary parts. Qaloon, Warsh the same. Warsh with its 144 MB is even exceeding the size limit by almost factor 5. If you say the one already present in my repo has the same libs: True. But had it as many when it entered my repo? I doubt that. I do not take in apps with 4+ such libs – and I do remove apps once they exceed 5 (given the authors a reasonable time to work on slimming them down – more than once such stuff slipped in unintentionally via dependencies).
That said, I'd appreciate the slim down for the one already in my repo as well :wink:
the naskh/warsh/qaloon ones have critical pieces that aren't open source so you couldn't add those easily I guess anyway. in the future, we're hoping to merge them all into one app anyway. the normal app should have Firebase/Crashlytics as toggleable from a build perspective.
the normal app should have Firebase/Crashlytics as toggleable from a build perspective.
So you could attach a "noanalytics" variant without too much efforts needed? That would be wonderful, and bring down the list of offenders to 3 or less (the other 3 are potentially only dependencies of those two). I'd then pick that for my repo, while you still could deploy the other one to Play.
Yes, should be able to - I just double checked and while analytics actually has a noop module for it as part of the open source, the same is not true for Crashlytics. I can make it optional, but then builds that go through this version won't have crash reporting at all. Does it make sense to use Bugsnag instead for this?
If you think you really need analytics, Bugsnag at least is FOSS (MIT licensed). It would still carry the Tracking
anti-feature, though. See here for some analytics options which are considered to be more privacy friendly (though not all of them have been checked in-depth).
Based on #2286, the app should be ready for f-droid, right?
Huh? Latest version still shows the same trackers and proprietary bits (Crashlytics, Firebase Analytics etc), so I'd say not ready. There was no new release yet with them removed.
@IzzySoft the same one from the other thread from some time back - the quran-3.4.4-no-google.apk
has no trackers. i thought you had set up to always pick up this pattern (though i haven't released any new version after then one you last checked - 3.4.4).
ApkMatch: /quran.*no-google\.apk$/i
looks like that pattern was set up after 3.4.4 was already pulled (you can see the different file date). I've pulled it manually now (size differs from the one here) and checked it: yes, that indeed looks OK. Replaced the APK here and removed the anti-features. Thanks for the heads-up!
quran-3.4.4-no-google.apk was built from which commit?
I did this on February 1st, which was just a very small amount of time after version 3.4.4 was released - my guess is, v3.4.4 tag built with ./gradlew assembleMadaniRelease -PdisableFirebase
should replicate that apk.
Most likely rather ./gradlew assembleMadaniRelease -PdisableFirebase -PdisableCrashlytics
. But still, quite a diff in the APKs:
-------------------------------
--- /dev/fd/63 2024-07-12 01:41:12.598563394 +0200
+++ /dev/fd/62 2024-07-12 01:41:12.598563394 +0200
@@ -1,19 +1,17 @@
META-INF/com/android/build/gradle/app-metadata.properties
32-bit CRC value (hex): 05cd8676
assets/dexopt/baseline.prof
- 32-bit CRC value (hex): a32cc6df
+ 32-bit CRC value (hex): 6c8a2597
assets/dexopt/baseline.profm
- 32-bit CRC value (hex): 02272ec9
+ 32-bit CRC value (hex): 1895f34d
classes.dex
- 32-bit CRC value (hex): a397c8ac
+ 32-bit CRC value (hex): 997acb10
assets/OpenDyslexic.otf
32-bit CRC value (hex): cefe3119
assets/UthmanTN1Ver10.otf
32-bit CRC value (hex): 54232161
assets/kitab.ttf
32-bit CRC value (hex): c3e6c290
- assets/quran.ar.uthmani.v2.db.zip
- 32-bit CRC value (hex): 1cded799
assets/uthmanic_hafs_ver12.otf
32-bit CRC value (hex): 637381de
DebugProbesKt.bin
@@ -184,12 +182,12 @@
32-bit CRC value (hex): 8f7d2509
META-INF/kotlinx_coroutines_core.version
32-bit CRC value (hex): 8f7d2509
- META-INF/services/sd.w
- 32-bit CRC value (hex): fbf95a97
+ META-INF/services/ad.n
+ 32-bit CRC value (hex): 0aa34b3d
META-INF/services/sun.net.spi.nameservice.NameServiceDescriptor
32-bit CRC value (hex): d7ca5a4a
- META-INF/services/xd.n
- 32-bit CRC value (hex): d0d40954
+ META-INF/services/vc.w
+ 32-bit CRC value (hex): 218e18fe
kotlin-tooling-metadata.json
32-bit CRC value (hex): 3ef3542a
kotlin/annotation/annotation.kotlin_builtins
@@ -211,9 +209,9 @@
...
And that's just the start of it. Walking the culprits in their proper order:
assets/quran.ar.uthmani.v2.db.zip
probably should not have been in your APK, so I guess you did not build from a "clean tree"?classes.dex
differs. That Dex diff is rather huge, and I'm no specialist in reading it – but it looks like your APK has things which are missing at the commit – and other things seem just to differ in ordering.baseline.prof
most likely only differs because of classes.dex
above (it includes the hash of the latter, that's why)META-INF/services/*
are easy, that's most likely just line endings (you build on Windows, right? We build on Linux), can be fixed on our ends.Looking at the Dex diff, here are some indicators of things missing:
|: invoke-direct {v0, v1, v2}, Ljava/lang/Enum;.<init>:(Ljava/lang/String;I)V
-|: sput-object v0, La4/a;.l:La4/a;
|: new-instance v1, La4/a;
|: const-string v3, "PENALTY_DEATH"
|: const/4 v4, #int 1 // #1
|: invoke-direct {v1, v3, v4}, Ljava/lang/Enum;.<init>:(Ljava/lang/String;I)V
-|: sput-object v1, La4/a;.m:La4/a;
|: new-instance v3, La4/a;
Ordering:
Static fields -
#0 : (in La4/a;)
- name : 'l'
+ name : 'j'
type : 'La4/a;'
access : 0x4019 (PUBLIC STATIC FINAL ENUM)
#1 : (in La4/a;)
- name : 'm'
+ name : 'k'
type : 'La4/a;'
access : 0x4019 (PUBLIC STATIC FINAL ENUM)
#2 : (in La4/a;)
- name : 'n'
+ name : 'l'
type : 'La4/a;'
access : 0x4019 (PUBLIC STATIC FINAL ENUM)
my guess is, v3.4.4 tag built with
./gradlew assembleMadaniRelease -PdisableFirebase
should replicate that apk.
My build was from that tag, though also disabling Crashlytics. So to narrow things down, I'd suggest the following steps:
./gradlew assembleMadaniRelease -PdisableFirebase
or ./gradlew assembleMadaniRelease -PdisableFirebase -PdisableCrashlytics
(the latter is what your repo here says, I took that from here)That given, I could run another try and let see the outcome, then let you know, @ahmedre. Deal?
Thanks @IzzySoft for the very detailed analysis! I can shed some light on what's happening here and also ask your suggestion / thoughts on proceeding.
First, I was checking, and despite it being confusing (I should fix it), -PdisableFirebase
and -PdisableCrashlytics
are the same thing (i.e. if one is disabled, the other is disabled as well). Will fix it at some point to just be one flag to avoid being confusing.
With respect to the change, this is expected - let me explain the current situation, what I am working on, and then leave you with a question or two.
Due to lots of people forking the code, adding ad banners and re-publishing the app (while keeping the use of our servers, increasing costs for us, getting us angry emails, and making money off of work that isn't theirs), I closed the app for a while many years back, and then re-opened it and kept an optional repository that was initially intended to be data only. The intention was just to make it slightly more difficult for people to do these types of blatant work, and it seems to be working decently well over the years. As time went on, however, new features got added into that optional (closed source) repository.
The code compiles fine without the private repository (but missing a bunch of data, features, etc), but with the private repository, the app is the complete one I publish on Google Play.
Recently, I've been doing a lot of work to narrow this gap so that the only extra stuff in the private repo is just literally data (a set of URLs, reciters, page types, etc) and that all other code is open source. You can see some of those patches in the recent set of patches pushed, and I plan to push more soon.
As an example, the assets/quran.ar.uthmani.v2.db.zip
is a downloadable file that the app will download if it is not shipped with the apk. At some point, I added it in the apk and copied it from assets to avoid everyone having to download it. This code only went in the private repo, however.
So now I turn the question to you - is the intention to only add 100% open source apps, or are you ok with minor variants (a class of Urls for example that's closed source) being built into the app? If the former, then I will give you a -no-google-open
build of sorts that is exactly what you'd build if you were to build it yourself with the -PdisableFirebase
flag. If the latter is ok to consider, then please give me some more time as I've started doing this work to narrow the gap of what's open and what isn't, such that at some point, my goal is for a few small data classes to be closed and for the rest of the code to be open.
Thanks!
So now I turn the question to you - is the intention to only add 100% open source apps, or are you ok with minor variants (a class of Urls for example that's closed source) being built into the app? If the former, then I will give you a
-no-google-open
build of sorts that is exactly what you'd build if you were to build it yourself with the-PdisableFirebase
flag. If the latter is ok to consider, then please give me some more time as I've started doing this work to narrow the gap of what's open and what isn't, such that at some point, my goal is for a few small data classes to be closed and for the rest of the code to be open.
I don't intend to respond in place of IzzySoft, but I'd guess 100% open source is what's we're looking for.
Recently, I've been doing a lot of work to narrow this gap so that the only extra stuff in the private repo is just literally data (a set of URLs, reciters, page types, etc) and that all other code is open source. You can see some of those patches in the recent set of patches pushed, and I plan to push more soon.
I noticed this and it's totally understandable, but I have a small suggestion. Since the "extra stuff" is effectively data, maybe it is better to keep it as metadata files, say json, sqlite, or anything else. This way you can keep one repo that is 100% open, while keeping the extra stuff hidden. You can then either inject that data when compiling the non-free version, or keep it on your severs encrypted with an obfuscated key that only exists in the non-free version. Personally I prefer the latter as it gives more flexibility.
One more point I'd like to bring again, which is the server. I do think that for an app to be fully free it should have an option to use servers which are fully free too and can be deployed by anyone. It appears that most (if not all?) of your server side code is available on github. I'm not sure about this. If you provide one build system, script, docker container, ansible playbook, or anything similar, for the entire server code and data, I'd gladly host a mirror myself and maintain it. And probably many others would quickly contribute. This would reduce your costs and make everyone happy. Just another suggestion, let me know what do you think.
Thanks
Thanks for the details, @ahmedre – and yes, fully understandable! The only part making me shake my head is how one can do such a thing (parasite a Q'uran app by adding ad banners) without being ashamed – but yeah, some folks have no respect…
Thanks also for explaining the -P
arameters – and yes, simplifying it is a good idea!
is the intention to only add 100% open source apps
As @Naheel-Azawy correctly puts it (full ack to the entire comment): yes, that's the intention. While at IzzyOnDroid, some non-free parts are tolerated if they cannot be avoided, bein fully open-source is always preferred. The "enhanced variant" can of course be pointed to e.g. from the app description. And could even be made available from within the FOSS app e.g. using some kind of key you provide (if I understood correctly, we speak about data here and not binaries) e.g. the way Naheel described.
Further, the -no-google-open
variant you describe could be made reproducible, while the other one cannot (as the verification builder would always lack the non-open parts).
At some point, I added it in the apk and copied it from assets to avoid everyone having to download it.
Which is of course totally fine – but then must happen with the build recipe, if you want the app to be RB. Else it's not part of the APK the verification builder produces, and thus leads to the app being non-RB.
For transparency, I've just added the NonFreeAssets
anti-feature to your app at IoD, outlining it contains data from a private repository (effective with the next sync around 6 pm UTC). That can (and will) of course be dropped once we switched to the -no-google-open
variant.
That said, maybe you want to complement your row of badges in the readme and pick a badge to link to your app at IzzyOnDroid? :smiley:
Thanks @Naheel-Azawy and @IzzySoft for the replies!
My plan of action:
disableFirebase
and disableCrashlytics
flagsWhen I have an update, can then share what the exact differences look like, and you can run another comparison and see if we need two flavors (100% open source and one that's 99% open source + non-open source data) or can just stick to one.
Thanks!
Thanks @IzzySoft and @ahmedre.
Sound like a good plan @ahmedre. And yeah php is not as shiny as it used to be, but I gotta confess, I still sometimes use it no shame in that. But don't tell anyone.
I'll be waiting for your updates. Once you find it ready, I'll be happy to host the server.
assalamu alaikum
May Allah reward all those who worked on this great software for their efforts.
Truly, a great piece of software, mashalah.
I recommended this software to f-droid.org, which is an android repository of open source apps.
They are strict about the apps being open source and free from any closed source blob, and they gave me this reply.
_"We have the metadata for this app since 2013-03-01, but unfortunately, we are unable to build. While the licensing issue has been fixed, it still uses non-free crashreporting software. They have a switch to disable it being used (which would be acceptable), but it still is required for building the app (which is not)."_
This is the link to the mentioned post: https://f-droid.org/forums/topic/quran-for-android/
Can anyone help with this "small" issue :-)
Jazakum allah khair wa alaikum assalam