supplementary location services based on local database downloaded from a server (not based on location history)

[thestinger]

These databases can be quite large, and they could have inaccurate data resulting in an initially inaccurate location fix before GPS overrides it, so this should be optional.

[cwmke]

Have you considered something like Mozilla Location Services? That way it wouldn't need the local storage.

https://github.com/microg/IchnaeaNlpBackend

[thestinger]

I think I'd want it to be local, at least for the main option.

[renlord]

something like this? https://f-droid.org/en/packages/org.fitchfamily.android.dejavu/

[OlafPettersson]

What would best approach here be? Both these plugins to microg's Unified Location Provider can do offline locations:

• https://f-droid.org/en/packages/com.google.android.gms/
• https://f-droid.org/en/packages/org.openbmap.unifiedNlp/

Do I understand correctly, that MicroG will not be included on GrapheneOS? If that's the case, one way forward might be to port microg's Unified Location Provider to GrapheneOS, and then talk to the aforementioned packages (and any others the user might desire to install, e.g. the geocoding package, solving #23 as well. Would this be a viable way?

[thestinger]

The implementation needs to be a lot more minimal and robust.

[OlafPettersson]

Can you elaborate on minimal and robust? To me, from a security standpoint, it seems like a reasonable setup: a higher privileged location provider, and some lower privileged actual implementations.

I can't vouch for the robustness of the above-mentioned plugins. If nessessary they probably can be rewritten more robustly. But what stands against using the same interface to those plugins - or replacements thereof - than microg? If a rewrite turns out to be desired or neccessary that would also allow to re-use these plugins in microg.

[Mynacol]

Data Sources

Only downloadable databases are desired. Databases used by existing UnifiedNLP backends are:

source	cell tower	WiFi	file format	size (approx.)	license
openBmap	y	y	sqlite	2.1 GB	Creative Commons Attribution-ShareAlike 3.0 Unported (cellular) and Open Database License (WiFi)
Mozilla Location Services	y	n	CSV	250 MB	Public Domain
OpenCellid	y	n	CSV	980 MB	CC BY-SA 4.0

openBmap includes opencellid data (according to the license page), and I guess Mozilla data as well. The last update was from 2020-02-16. Mozilla and OpenCellid data is smaller, only has cell tower data, and is provided as well-defined CSV. The data should probably be converted to sqlite to be indexable.

FastLacellsGenerator combines all three to one big sqlite database to be used by Local GSM Backend. This could be done as part of the system building process.

Architecture

I think the plugin-based system of UnifiedNLP is not a bad idea. This way we can easily add support for other location providers (WiFi, Bluetooth beacons). If AOSP already supports multiple (coarse) location providers at the same time we can drop this idea. Some considerations: The backend (de-)registration is unneeded. We can hard-depend on the shipped backend(s). This removes some intents, and the other intents can only be allowed by the defined system backend(s). So no (user) app can get new intents/resorces/permissions. The GUI/Settings and download stuff can be removed if we ship the database with the system image. Then the app focuses on location.

Database deployment

Bundling the database with system images dramatically increases their size. The main alternative would be a separate download, ideally signed. Then we need a GUI for it, explaining users why they would want it etc. So for now bundling with the OS is the easiest method.

I assume we would ship data for the whole world, and not selecting specific countries.

[thestinger]

A plugin-based model without enforced signature verification bypasses the intended security model. There are also issues with hot this is implemented. It's not going to be implemented that way. We're also definitely not including microG due to severe security issues with it. The only way this will be included is as a minimal and robust app. It should have no reason to be privileged beyond being whitelisted as a location provider. The privilege separation that you're talking about not only doesn't make sense but as implemented breaches the security model by bypassing the whitelist of providers.

[thestinger]

Also, please note that anything licensed as GPLv3 or AGPLv3 also cannot be included in GrapheneOS.

The desired approach is implementing a location provider as an app where our releases of it are whitelisted. We want to have one that's included by default and has a simple toggle for enabling it and beginning to keep the database updated.

AOSP already supports whitelisting multiple location providers. Unless there's a good reason not to use the standard AOSP system for this, that's what we'll want to use. We don't want to have a central app that's whitelisted and which needs to implement a whole separate setup for whitelisting plugins. It makes more sense to use the existing system as long as it works properly for the desired implementation.

This should have nothing to do with a compatibility layer for Play services and we're never going to have any kind of support for microG. There will be a stubbed out Play services compatibility layer as a disabled by default option available to users as described in https://grapheneos.org/#never-google-services but that has nothing to do with this. Geocoding as mentioned in #23 also has nothing to do with Play services. It is an AOSP API requiring a provider, just like this, and needs to be implemented in an appropriate way.

[OlafPettersson]

From a feature, license and maintenance perspective https://gitlab.com/deveee/Local-GSM-Backend seems to be a reasonable starting point. The implementation supports both mozilla location service and opencellid, downloads the data into a local sqlite db and seems to have a robust location algorithm (in TelephonyHelper.java). It is licensed under the Apache License and seems to be well maintained.

I'm not sure what would be the best way to integrate the package into GrapheneOS as µG is not an option.

In any case, I'm not sure a single configuration switch will suffice, since the amout of data that needs to be downloaded regulary is quite large. Also, most users probably want to select only a few countries for which to download and update GMS data.

[OlafPettersson]

tl;dr: For offline-location based on cell data from opencellid.org and Mozilla Location Service, only a very rough location estimate with accuracy between 500m and 10km is achievable. WiFi data is not available from non-proprietary sources for offline usage.

To get a better grip on where the open source community stands with network location providers on android, I wrote a simple app, "Location Tester". It instantiates and calls UnifiedNlp-Plugins by their very simple API. The app does not need MicroG or their UnifiedNlp-Service installed.

Here are my observations:

Name	License Source Code	Operation	Remarks
LocalGsmNlpBackend org.fitchfamily.android.gsmlocation	Apache License 2.0	Offline / GSM/UMTS/LTE	Uses an offline database blended from opencellid.org and Mozilla Location Service (MLS). Can download directly from these services or from a repository at github. As for the locations, for me, results are mixed. The phones I have available only report the stongest network cell, not neighboring cells. Triangulation is thus impossible. The strongest cell near my home is not mapped by either opencellid or MLS. When in other places, I do get locations from this backend. These locations are often both off and have a quite large reported confidence area of 1000-5000m. The true location is most of the time in the reported confidence area. As a side note, as far as I can see, the returned confidence radius does not take into account the definition differences of Android (68% confidence) with ranges returned from MLS (some kind of "100%" confidence) or from opencellid (98% confidence).
Déjà Vu org.fitchfamily.android.dejavu	GPL3	Offline / WiFi/GSM/UMTS/LTE	UnifiedNlp backend that uses locally acquired WLAN/Wi-Fi AP and mobile/cellular tower data to resolve user location. This backend uses no network data. All data acquired by the phone stays on the phone. I found that this gives very good results when able to learn from an enabled GPS. I believe it might be a problem for GrapheneOS that this backend basically builds an on-phone database of visited locations. If you would ever have to hand over your phone-data, you would have to hand over where you have been.
LocalWifiNlpBackend org.fitchfamily.android.wifi_backend	GPL3	Offline / WiFi	Superseeded by Déjà Vu backend. Works only with WiFi-data. Gives quite good results when able to learn from enabled GPS.
Radiocells.org UnifiedNlp Backend org.openbmap.unifiedNlp	AGPL3	Offline/Online / WiFi/cell	Radiocells.org seems to be put on hibernation. The latest data update appears to be from January 2020. I could get online mode to work, but not offline mode.
MozillaNlpBackend org.microg.nlp.backend.ichnaea	Apache License 2.0	Online / WiFi/GSM/UMTS/LTE	You hand your IP, WiFi and cell data to Mozilla and get a location. Can be configured to only use WiFi or cell. For me results are ok to mixed. The reported location/confidence sometimes appears to be wrong/too small when using WiFi. It is large (>1000m) but mostly correct when using cell. Mozilla does not share WiFi data for offline use.
Apple UnifiedNlp Backend org.microg.nlp.backend.apple	Apache License 2.0	Online / WiFi	I did not test this myself. But what I have seen is, when indoors or in developed areas, this gives very accurate results. The reported confidence area appears to be larger than needs to be. Obviously, everytime you use this plugin, you hand over your IP address and location to Apple.

I am not sure how to go forward with this. I find the GNSS on my Pixel4a quite quick and accurate. I am not so sure any of the non-proprietary data sources can add much value to it.