Closed IzzySoft closed 1 year ago
F-Droid can maintain a patch disabling this and remove functionality in few quest relying on it which measure width of some objects.
There is already fallback to measuring width with a tape and entering data manually.
If SC can be changed without making it worse in way that makes maintaining patch easier, I guess that they can submit such pull request.
Or if there is a drop-in replacement for this library (afaik no) that also would be nice to know.
I've seen and asked about this in https://github.com/streetcomplete/StreetComplete/pull/3709 at that time and was at ease since it said "but it's foss"
But now looking at https://github.com/google-ar/arcore-android-sdk and comparing with the AAR served by maven https://mvnrepository.com/artifact/com.google.ar/core/1.32.0 which has .so's and classes and whatnot, I fail to see WHERE is the source code exactly. :(
/LE: I've disabled the affected versions for now: https://gitlab.com/fdroid/fdroiddata/-/commit/94c0447a535e3ceed3cd41dd25d83b4666b378b3
Thanks for the fast answer! Indeed I'm not aware of any "smooth replacement" either (but I'm no dev, so I might simply have not heard of it). Thanks for the detail that it's at least not "crucially essential"! @licaon-kter maybe you could take a look at how to achieve that, so builds can be re-enabled?
Adding some references for open AR replacements: https://github.com/streetcomplete/StreetComplete/issues/2093#issuecomment-1090606995 https://github.com/microg/GmsCore/issues/1672
@IzzySoft so which consequences does this scan have at all?
I'd not like to push an F-Droid version of the app that has the AR feature disabled. Only because one installed the app from F-Droid shouldn't mean that the choice is taken away from the user whether he wants to install a non-free app to enable the AR feature or not. As HolgerJeromin linked in the previous comment, you do not even need Google Play Services as you may be able to sideload the ARCore app from another source.
To clarify:
com.google.ar:core
and that dependency itself is open source, see https://github.com/google-ar/arcore-android-sdk. So maybe that warning is imprecise(?)@westnordost can you read my comment above pls? This issue is not about the separate app.
I fail to see WHERE is the source code exactly. :(
It is on github. Certainly not in the AAR, I do not think an AAR is meant to contain source code.
Hmm right... there is not a lot in that repository, most of all, no java code.
@westnordost thanks for the link to the AR repo. But please take a look at the license there:
Governed by the ARCore Additional Terms of Service at https://developers.google.com/ar/develop/terms
That's "source available" at best (if the code would be there, as you just found out), but not FOSS.¹ And as you wrote yourself, you just include the AAR. F-Droid needs to build it from source (to make sure it matches the source code presented – which certainly would fail here I guess if the source repo does not contain the source).
If I remember correctly, not even the button to start the AR measurement will be shown.
So maybe as a "quick fix" we can set up a rule in the build recipe to "patch out" the two implentation lines, to at least keep the app listed at all until we found a final/better approach?
¹ I just see that applies to the pre-compiled binaries – but iif I understood correctly that's what you use by including the AAR? The code itself seems to be Apacke-2.0 – if we can find (and compile) it…
On the other hand, com.google.ar:core
is listed as licensed under Apache License, Version 2.0
on Maven Repository because in the POM file it is stated to be that license by the library author (Google LLC). So either Google violates their own terms in not also publishing the source code or they have hidden it really well.
If they want their additional terms of service to apply to that Android library published via Maven too, then they should (have) stated that in the POM.
But I feel there is some missing information here. That
* Google ARCode (/com/google/ar/core): NonFreeDep
is flagged can only come from that someone added that rule manually in some tool, right? So, how come it was added? Maybe a rationale can be found in the commit message (in the repo for whatever tool is this).
Anyway, at the moment I am not inclined to create a special version for F-Droid without this dependency. You'd have to do this yourself, albeit it is not a lot of work to create a patch.
The reason is that I don't accept that the (alleged) failure to comply with their own open source license of some dependency this app is using should lead to expulsion of this app from the F-Droid. As far as I am concerned, for me it's enough to know that the library is open source on paper, because that means that the app including all its dependencies may legally be distributed under these terms, i.e. including decompilation and modification of the code in e.g. the com.google.ar:core
dependency even if the source code of that is nowhere to be found.
Nevermind my statement about the POM, I noticed that while it says Apache License, Version 2.0 in the name, it actually links to the LICENSE file in the repo. However, in that LICENSE file in the repo, they make clear that the additional terms of service only apply to
, with the rest being Apache License, Version 2.0. The AAR published via maven does not contain any of these.
F-Droid needs to build it from source (to make sure it matches the source code presented – which certainly would fail here I guess if the source repo does not contain the source)
As far as I know, F-Droid builds with gradle build
, which does not build every dependency itself but (just) download all the dependencies from maven central and google (i.e. the JARs and AARs with precompiled .class files).
It's all fun and games until I ask again about the source? I've got nothing to build...
How did it work before up until v45.2 then?
How is it that we didn't scan for this dependency before?
As usual, not enough hours in a day for contributors to port nonfreedep
from one source ( https://gitlab.com/IzzyOnDroid/repo/-/tree/master/lib ) say via grep -i nonfreedep libinfo.jsonl|cut -d ":" -f2|cut -d "\"" -f2|cut -d "/" -f2-|sed 's!/!.!g'
to the fdroidserver scanner ( https://gitlab.com/fdroid/fdroidserver/-/blob/master/fdroidserver/scanner.py#L62 ).
Blacklisting bad stuff is a never ending war, there's always one more.
/rant @eighthave maybe merge some of the "scanner" MRs that await too? https://gitlab.com/fdroid/fdroidserver/-/merge_requests
That was not my question. How did it build before if allegedly the F-Droid build bot builds each dependency itself from source instead of just the app from source? (That's what I understood you implied in https://github.com/streetcomplete/StreetComplete/issues/4289#issuecomment-1216865343)
We don't build everything from source, see: https://f-droid.org/en/docs/Inclusion_Policy/ (starting with Though we tried to build everyting from source).
As it would be a wasted effort when things are build already from source. That's the issue here, this dep is hosted on a "trusted" maven repo but they didn't do due dilligence to verify if truly FOSS. Not an isolated case. More info in this future post: https://gitlab.com/fdroid/fdroid-website/-/merge_requests/838/diffs
@westnordost I'd gladly state that I was wrong all along, and apologize in any way – if only the source were there. Until now, all we have is a binary AAR file and a statement that it's supposed to be Apache-2.0 – a license that states we "may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form" – thus granting us the freedoms required to be counted as "F/LOSS". But we cannot make use of that freedom if the source is not even there. There's not even the freedom to investigate the source, making sure what it does and that it corresponds to the AAR – if the source cannot be found.
All I see in google-ar/arcore-android-sdk are some media files, C headers – and example code. But not the source code of this AAR. You've seen that for yourself. So if the source is not available, it cannot be "open source" – and thus cannot be accepted by F-Droid :cry: And no, I don't see any "alternatives" behind the links @HolgerJeromin posted: if I didn't miss something essential, they basically state that you can use AR with StreetComplete if you simply sideload that APK. Which is only possible because StreetComplete includes this AAR.
I fully understand your frustration, Tobias – and believe me or not, I fully share it. I just started using SC a little more than a month ago, and already I am so fond of it that I recommend it everywhere, motivating people to use it. I even made something I didn't do since my repo exists: I set up to have it serve the expert edition, despite of its being more than two times over the size limit (I've never let an app enter my repo that way – that's rather the time I unlist an app). So I very much hope we can find a solution here. Ignoring the fact doesn't count as such (we cannot do that) – so it's either we turn up the source, or we need to "outsource" it somehow.
Now I thought about asking them at their repo – they won't accept that: all questions shall be asked at SO using the ARCore
tag. Searching the tag along with the keyword "source", I've stumbled upon Is Sceneform 1.17.1 open source? – which states:
As it is identical to 1.15.0 which is closed source, and the source is not provided separately, it is probably safest to assume it is actually closed
Looking into the smali of SC, there is /smali/com/google/ar/sceneform
– so where does this come from? AAR from the same source? For this at least there seems to be an open source variant available: sceneform-android, but it unfortunately seems to depend on the very same ARCode we cannot find the source for (though it seems to be optional if you only need Sceneform itself). I could not find an answer to the sources of Core, though, nor did a web search succeed (just found the same repo, Google's docs, and plenty of how-to-use).
So if we cannot find the source, and neither a FOSS replacement: can that maybe moved to an addon entirely, so the app itself stays FOSS and thus at F-Droid?
PS: For what it's worth, I started a shout at Mastodon. Let's see if it turns something up…
Sceneform is a separate library and is not a problem. The code is available.
@licaon-kter: so that makes this a question of policy rather than a question of necessity. You do not want to have any apps in F-Droid whose dependencies you could not theoretically build from source regardless of whether they are licensed with an open source license or not.
I've read through the Apache license 2.0 and it looks like indeed the licensor does not commit himself to also publish the source, there is no mention of it. Hence, to my understanding, a work can be licensed with Apache 2.0 without the source to be available.
However, this does not seem to touch on the rights of the licensee to produce derivative work, in other words, decompile it, make modifications, redistribute under the same terms:
Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
My point being: the licensee has every right to decompile the bytecode, make modifications and redistribute it, and that's what free software is about, that you are granted the right to do it. It is not about the ease of doing so. (An obvious flaw in non-GPL licenses)
Hence, I say that this a question of policy, i.e you do not want to include any apps whose dependencies do not also include the (human readable) source code in F-Droid. Regardless of the license.
Assuming that the apparent non-inclusion of the source code in the Apache licensed work was even a deliberate act rather than an oversight on the basis of "meh, don't care" by Google ARCode devs, wouldn't it solve the policy issue if a decompiled version of that lib was available somewhere as a "derivative work"?
I've read through the Apache license 2.0 and it looks like indeed the licensor does not commit himself to also publish the source, there is no mention of it. Hence, to my understanding, a work can be licensed with Apache 2.0 without the source to be available.
Yes, it can. It is then however neither free software (i.e. lacks "preferred form of the program for making changes in") nor open source (i.e. is not "software with source code that anyone can inspect, modify, and enhance.") - license notwithstanding (kind of like Microsoft Windows is not free software nor open source software - although it contains, in binary form, significant amount of BSD-licensed code, and Microsoft Windows do show you all those licenses)
F-droid is a repository of "free and open source software (FOSS)". If software is not FOSS, it cannot (should not) be included there. And yes, that is because of their FOSS-only policy (which I personally happen to wholeheartedly approve of), which is arguably only reason for it's existence.
"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
What I understand from that is that as far as the license is concerned, the compiled blob of bytecode is the "preferred form for (providing for) making modifications" by the licensor, as there is no alternative
No, I'm pretty confident you misunderstood there. Source is "preferred form for making modifications" -- not "only form actually available publicly" or anything like that. So unless the original authors of software made that .aar
library by typing numbers in hex editor, but preferred some other way of creating it, than that other way is actual preferred form of making modifications, and thus the Source is that what original authors used. Even minified javascript (for example) is not source code - because it is not the preferred way to write javascript.
My point being: the licensee has every right to decompile the bytecode, make modifications and redistribute it, and that's what free software is about, that you are granted the right to do it. It is not about the ease of doing so.
Well, it's not what free software is about, at least if you ask Free Software Foundation (/ RMS) who coined that term "free software". See link above, specifically part starting with Therefore, accessibility of source code is a necessary condition for free software. Obfuscated “source code” is not real source code and does not count as source code.
Hence, I say that this a question of policy, i.e you do not want to include any apps whose dependencies do not also include the (human readable) source code in F-Droid. Regardless of the license.
Yes, free software / open source / FOSS / f-droid is always about policy, regardless of license. In fact, you can have code which does not have _any license at all_ (for example, because it is Public Domain, i.e. not covered by Copyright and related rights, and thus not needing license to relinquish that copyrights) - and it can still be free software and open source and included in f-droid.
Assuming that the apparent non-inclusion of the source code in the Apache licensed work was even a deliberate act rather than an oversight on the basis of "meh, don't care" by Google ARCode devs, wouldn't it solve the policy issue if a decompiled version of that lib was available somewhere as a "derivative work"?
Uhhh, I wouldn't really want to venture there, nor have people I've become fond of paint such targets on their back for google lawyer-army to practice on, even if they can prove beyond doubt that that binary package indeed corresponds in full to that abandoned github page, and that it's indeed intended to be its license (which is far from conclusive).
in fact, you can have code which does not have any license at all (for example, because it is Public Domain, i.e. not covered by Copyright and related rights
note that in this case it is better described as being licensed as public domain, not as having no license at all
in case of not having any license at all and not having copyright waived by law - works are fully copyrighted
Looking into the smali of SC, there is /smali/com/google/ar/sceneform – so where does this come from?
I am confused by https://github.com/google-ar/arcore-android-sdk/issues/1049 - is sceneform being
@matkoniecz If I read that correctly, Sceneform was abandoned. The last thing they did was fully open-source it¹, then archive its repo. There's a sill-alive and community-maintained Sceneform fork available, though.
¹ v.15 was closed source, 1.16 open-source with parts missing, v1.17 then added missing source parts
My toot on Mastodon meanwhile brought in some answers. Basically, nobody seems to be able to find the full source; even Marvin W (author of microG) states it is incomplete. Another dev will raise the question in Google's issue tracker today; let's hope we get a positive answer there.
v1.17 then added missing source parts (...) nobody seems to be able to find the full source
So StreetComplete is using an abandoned project that was open sourced - but in way that still left part of code not actually published?
In such case I would at least try asking to publish also missing parts. So "another dev will raise the question in Google's issue tracker today" seems a good next step.
Here's the link to the issue: https://issuetracker.google.com/issues/242730604 (whoever has a Google account can give it a +1 or chime in).
I guess that it would be helpful to point out specific missing parts, if any were identified to be missing. Right now it looks like support request likely to be ignored.
It would be better to have "ARCore-Android-Sdk is described as open source but class XYZ is only in binary blobs, and not available is source code on XYZ repository, please publish also missing parts"
even Marvin W (author of microG) states it is incomplete
link goes to some general repo, not to statement confirming this
Hey guys,I found the sources:
https://dl.google.com/dl/android/maven2/com/google/ar/core/1.32.0/core-1.32.0-sources.jar
When clicking the "View all" link on https://mvnrepository.com/artifact/com.google.ar/core/1.32.0 , the Google repo returns 404 on the directory listing. However, the naming scheme is always the same in maven, so I guessed the name.
The top message of the .java files is fun:
/*
* Copyright 2018 Google LLC
* CONFIDENTIAL. AUTHORIZED USE ONLY. DO NOT REDISTRIBUTE.
*/
(╯°□°)╯︵ ┻━┻
Okay, the dev(s) just confirmed even though the Java stuff is actually Apache licensed 2.0 and the above mentioned copyright notice is a "bug", there is a binary blob within that AAR which is proprietary, making the library as a whole not open source.
So, AR functionality must be removed for F-Droid publication.
But it still can be used in SC itself, right? Is https://developers.google.com/ar/develop/terms conflicting with GPL for SC code in general?
FYI I plan to defer working on this to re-enable F-Droid builds because I like to first wait a bit to see if the Google folks at arcore-android-sdk have a change of heart. It doesn't seem to be a big team, maybe they can make such decisions on their own measure.
If anyone wants to see F-Droid builds pop up sooner again, you are welcome to create a PR. I am not sure how custom builds for F-Droid should be handled. Of course the PR must not remove the AR stuff, but just make it so that (a newly to be defined) build flavor for F-Droid does not include the AR-specific sources and dependencies. I am not sure how to do that, you'd need to read some gradle documentation. Or alternatively, maybe F-Droid folks offer another mechanism how to patch an app for the F-Droid build bot: Maybe it is enough to write a "patch" that automatically gets applied before each build? In either case, it necessitates some reading into documentation.
F-Droid support patching before building, yes.
@westnordost as re-establishing SC in F-Droid might take a bit then (@licaon-kter just meant to say the option exists, not that a patch is in the works) – if you think it's needed and would help, I'd make a second exception from the size-rule and offer SC an "interim home" in my repo until it returns to F-Droid.org. Signatures would not match, so it won't help existing F-Droid users (myself included) much unless they are ready to uninstall and reinstall and it might be easier for them staying with the installed version or (temporarily) use the extended edition (available via my repo) alongside – but the offer stands.
Speaking of the EE: can existing data from SC be transferred to SC EE (and back) in some way?
Nah, I do not think this would make sense. People can just use either the version from Google Play or from GitHub if they want the newest features. For EE, you'd need to ask @Helium314 if he changed any database tables or names in the shared preferences. If not, you can move the directories where the data lies accordingly, but you need to have full file system access to access the private folders of the apps (root, maybe adb?).
Thanks! I thought I might have missed an export/import feature (would be a different issue then, if you'd accept that feature request, as it'd help "migration").
for import/export see https://github.com/streetcomplete/StreetComplete/issues/3481
Database and preferences files can be transferred between EE and vanilla SC, only the TangramPinsSpriteSheet
preferences should be removed from preferences file, or quest pins/icons may be missing or messed up.
But it still can be used in SC itself, right?
No, not at all - i.e. not unless:
StreetComplete.apk
is supposed to be licensed under GPLv3. Section 6 "Conveying Non-Source Forms" states (among other things) "You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License"
As the source of the part of the .apk
is missing (no source for ARcore
at the moment), we are physically unable to "convey the machine-readable Corresponding Source" in any way - and thus may not "convey a covered work in object code" -- i.e. may not distribute affected .apk
files at all.
We may continue to distribute the source from https://github.com/streetcomplete/StreetComplete/ (which is all FOSS) and ask all users to compile .apk
files themselves (as GPLv3 restrict distribution only, not usage), but that would of course be highly inconvenient to 99% of the users (to say the least!)
So to continue distributing .apk
files anywhere (Google Play, GitHub, F-Droid etc) under current license we need to solve this issue:
StreetComplete-nonfree-ARcore.apk
plugin package - like OsmAnd used to do in times past for its Contours and Parking plugins for example), and only call the functions there if the users installed that plugin. The users who want to continue using AR functionality in quests would need to install that plugin.apk
package in addition to main SC.apk
package.Is the compiled StreetComplete.apk
really GPLv3-licensed? Or only the source code?
Uff, I didn't know that. I was under the impression that GPL would only be viral in that if a GPL licensed piece of code is used (e.g. a library), the (binary) distribution as a whole must also be GPL, not the other way round (e.g. it is not enough if all the code owned by the project is GPL, also all dependencies must be compatible to GPL).
This means that it is not even an F-Droid issue anymore. That Google's ARCore library contains non open-source code means that any copyleft-licensed project cannot use ARCore, distributed anywhere, which is not made transparent at all by Google (on the contrary).
I figure this creates a lot of work or even makes the work invested so far moot for any copyleft licensed open source project that has been lured into using or even basing their software on Google's false statement that the ARCore Android library is open source. I do not think there was a deliberate intent but it could surely be interpreted as feigning false facts.
The non open-source object in the library has an estimate of 2000 lines of code (60KB, see this paper) but this including any included dependencies as Abseil (, boost?, ...). So, given that the native part of that library likely only does the communication to the proprietary APK, it may be easy to reverse-engineer this. One may interpret that the license currently says that (at least) the object form of this native part is Apache licensed. However, I will not do this because 1. I noticed that not even the Java sources are complete. In the linked source JAR, many methods have their contents removed, 2. legal uncertainty regarding the reverse-engineering, 3. I want to wait on a whether they will consider open-sourcing the closed-source part of their library. 4. It is a lot of work
In any case, what I intend to do is:
Ugh, that becomes much heavier than I thought… So if case 3) should come true (which we all hope will not), how will you go for the F-Droid build? Reading between the lines ("a separate Apache-licensed app"), the current one will have ARCode stripped so we can build it again? Just asking for clarification. And thanks for all your efforts!!!
The entire point of such complicated splitting would be to eliminate tainted code from StreetComplete.
Though depending on what F-droid needs: is it ok to propose to user to install proprietary app? If no then F-droid would need to patch it away
NonFreeAdd
anti-feature?
That would earn you the NonFreeAdd anti-feature, yes. But it would be acceptable, yes.
Wouldn't it be the same already even if the Android ARCore library was open source? After all, it "promotes" (i.e. doesn't work without) the "Google Play Services for AR" APK and even offers a short-cut to install it AFAIK.
Yes and no. The description says "promotes or entirely depends on". So if it's just a possibility (as it currently is – at least I wasn't yet pointed to ARCode in the app), it would not necessarily raise the AF. Border might be "thin" or "arguable" – but that's how I would categorize it here. When in doubt I usually consult with e.g. @licaon-kter for a second opinion :wink:
I've just scanned the APK built by F-Droid, and see:
This would mean all affected builds need to be disabled, as that's a no-go for F-Droid. So the questions are:
apktool d de.westnordost.streetcomplete_4504.apk
), this does not look like a "stub". Am I mistaken?Being a StreetComplete user myself, and just having made a big announcement at Mastodon how useful the app is, I'd hate seeing it disabled at F-Droid – so I really, really hope you can find a way to mitigate this.