[1.2.5]"Show Nearby Emitters" triggers crash

[Mrnofish]

With Local NLP not being initialized yet. Crashes happen regardless if Wi-Fi Scanning is enabled under Location or not.

This is on A13 / LOS20.

I'm not seeing anything super obvious in logcat, except this fairly cryptic (for me at least) set of messages that tend to repeat:

E WifiHAL : get_wifi_interface_info: QCA_WLAN_VENDOR_ATTR_LL_STATS_IFACE_INFO_TS_DUTY_CYCLE not found
E WifiHAL : nl80211: requestResponse->nl_recvmsgs failed: -5
E WifiVendorHal: getWifiLinkLayerStats_1_6_Internal(l.1199) failed {.code = ERROR_INVALID_ARGS, .description = }

[Helium314]

Does the app have the location permissions? (and: did it ask for the permissions? There was a bug that should be fixed now, but not sure...)

Does it crash immediately when you click "Show Nearby Emitters", or after a few seconds?

[Mrnofish]

Most definitely, it only let me pick "while open" at first, but soon after, I was given the option to choose "all the time".

I think the crash is immediate because it drops back to the module's configuration page right away, but I only get the message "keeps crashing" if I insist.

[Helium314]

According to your description, the crash likely happens when a location is requested, or when trying to get the location service.

Since there is a crash, there should be a stack trace visible in logcat... did you see anything containing "org.fitchfamily.android.dejavu"? (the old name is still used internally)

[Helium314]

If the issue is getting the system service, it might be fixed in this debug build: local-nlp-backend_1.2.6-debug.zip

For anything else, I need some more info to understand what might be wrong.

[Mrnofish]

Yes I saw instances of that app string and was expecting a stack trace too, but didn't catch one.

Same thing with the debug version, I'll go sift through the logcat, let me get back to you.

[Mrnofish]

Got it:

E AndroidRuntime: FATAL EXCEPTION: main
E AndroidRuntime: Process: helium314.localbackend.debug, PID: 11676
E AndroidRuntime: java.lang.IllegalArgumentException: provider "network" does not exist
E AndroidRuntime:   at android.os.Parcel.createExceptionOrNull(Parcel.java:3015)
E AndroidRuntime:   at android.os.Parcel.createException(Parcel.java:2995)
E AndroidRuntime:   at android.os.Parcel.readException(Parcel.java:2978)
E AndroidRuntime:   at android.os.Parcel.readException(Parcel.java:2920)
E AndroidRuntime:   at android.location.ILocationManager$Stub$Proxy.getCurrentLocation(ILocationManager.java:1284)
E AndroidRuntime:   at android.location.LocationManager.getCurrentLocation(LocationManager.java:1008)
E AndroidRuntime:   at android.location.LocationManager.getCurrentLocation(LocationManager.java:916)
E AndroidRuntime:   at org.fitchfamily.android.dejavu.SettingsActivity.onClickScan(SettingsActivity.kt:484)
E AndroidRuntime:   at org.fitchfamily.android.dejavu.SettingsActivity.onCreate$lambda$4(SettingsActivity.kt:74)
E AndroidRuntime:   at org.fitchfamily.android.dejavu.SettingsActivity.$r8$lambda$FJ5Q6wY9e6nn-eqHxHxf6Dnw10g(Unknown Source:0)
E AndroidRuntime:   at org.fitchfamily.android.dejavu.SettingsActivity$$ExternalSyntheticLambda14.onPreferenceClick(Unknown Source:2)
E AndroidRuntime:   at android.preference.Preference.performClick(Preference.java:1165)
E AndroidRuntime:   at android.preference.PreferenceScreen.onItemClick(PreferenceScreen.java:261)
E AndroidRuntime:   at android.widget.AdapterView.performItemClick(AdapterView.java:330)
E AndroidRuntime:   at android.widget.AbsListView.performItemClick(AbsListView.java:1257)
E AndroidRuntime:   at android.widget.AbsListView$PerformClick.run(AbsListView.java:3270)
E AndroidRuntime:   at android.widget.AbsListView$3.run(AbsListView.java:4236)
E AndroidRuntime:   at android.os.Handler.handleCallback(Handler.java:942)
E AndroidRuntime:   at android.os.Handler.dispatchMessage(Handler.java:99)
E AndroidRuntime:   at android.os.Looper.loopOnce(Looper.java:201)
E AndroidRuntime:   at android.os.Looper.loop(Looper.java:288)
E AndroidRuntime:   at android.app.ActivityThread.main(ActivityThread.java:7884)
E AndroidRuntime:   at java.lang.reflect.Method.invoke(Native Method)
E AndroidRuntime:   at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
E AndroidRuntime:   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936)
E AndroidRuntime: Caused by: android.os.RemoteException: Remote stack trace:
E AndroidRuntime:   at com.android.internal.util.Preconditions.checkArgument(Preconditions.java:59)
E AndroidRuntime:   at com.android.server.location.LocationManagerService.getCurrentLocation(LocationManagerService.java:701)
E AndroidRuntime:   at android.location.ILocationManager$Stub.onTransact(ILocationManager.java:570)
E AndroidRuntime:   at android.os.Binder.execTransactInternal(Binder.java:1285)
E AndroidRuntime:   at android.os.Binder.execTransact(Binder.java:1244)
I ActivityManager: Killing 14385:helium314.localbackend.debug/u0a273 (adj 900): remove task
I WindowManager: WIN DEATH: Window{1022e97 u0 helium314.localbackend.debug/org.fitchfamily.android.dejavu.SettingsActivity}
W ActivityManager: setHasOverlayUi called on unknown pid: 14385
W UsageStatsService: Unexpected activity event reported! (helium314.localbackend.debug/org.fitchfamily.android.dejavu.SettingsActivity event : 23 instanceId : 197399478)

[Helium314]

Thanks! This is really weird though. I wasn't aware that it it's possible to have microG, but no network location provider.

Does LocalNLP (or any other location backends) actually work / provide locations?

[Helium314]

One more try. This should not crash, but may not work. If there is no network provider, it will try the fused provider, but I'm not sure whether this will work (even if it exists). local-nlp-backend_1.2.6-debug.zip

[Mrnofish]

It works! First it says it will try fused then a second toast informs that the scan has failed.

Bear with me because it's obvious now I have no idea what "network" actually means: I assumed it was provided by backends but I see that is something different and separate.

I've downloaded an application just for the sake of testing, which has separate switches for GPS, network and back ends.

It says network is unsupported but I don't know why. There is nothing inside MicroG about location other than the backend page, and nothing very helpful is in the system settings, so I'm not positive what should be doing about it?

The location warning comes up intermittently when the app is in foreground, like it's making repeated attempts, but no location seems to come up. I would expect wifi based location shouldn't take a long time to get a fix?

BTW I had imported the MLS database soon after installing LocalNLP, nothing seems to have budged.

Not sure if relevant, I have the GPS daemon blocked in the firewall, I thought that only provided the GPS receiver with assistance data, and the less I send to Qualcomm the better.

I'm closing this now as the crash is resolved but some help would be greatly appreciated!

[Mrnofish]

PS: I've installed Mozilla's NLP and divestos Merged WiFi. Mozilla is the only one working for me, merged OTOH imported the MLS DB so quickly I don't think it did anything actually.

Is root needed for db import to complete successfully in LocalNLP? The docs only say to extract.

[Helium314]

There is nothing inside MicroG about location other than the backend page, and nothing very helpful is in the system settings, so I'm not positive what should be doing about it?

The missing network provider is really strange, maybe it's an issue with microG?

I would expect wifi based location shouldn't take a long time to get a fix?

You can only get a location if the wifi is already in the database. For wifis, you can either import another database or export file if you have one, or fill the local database yourself. This is basically by having GPS enabled and at good accuracy near a wifi. But since you already imported MLS database, you should still get low accuracy location based on cell towers.

Not sure if relevant, I have the GPS daemon blocked in the firewall

That should not be relevant, as long as GPS is still working (for filling the database).

Is root needed for db import to complete successfully in LocalNLP?

No root needed at all.

Mozilla is the only one working for me.

So you get locations from the Mozilla backend?

[Mrnofish]

From what I understand the 'network' provider is actually data provided by the cellular network. So if you are on a network that doesn't give that information ...you're (pardon the French) SOL.

I'm getting fairly accurate location (dozens of meters) from the MLS backend using only wifi information (airplane mode on) in a reasonable timeframe, we're talking seconds.

I'm not sure why LocalNLP or other backends aren't producing any result, but it's doubtful that the network is responsible for that.

When I imported the MLS DB into LocalNLP, it seemed to work up to a point but towards the end the skipped counter started going and hit what seemed a fairly high number (hundreds of thousands IIRC).

I learned that you can get more detailed information from dunpsys location but haven't had time to take a quiet look at the logs yet.

Assuming i find anything, I will let you know, if you're interested.

[Mrnofish]

BTW 1.2.6 from fdroid is still crashing with the same error.

[Helium314]

When I imported the MLS DB into LocalNLP, it seemed to work up to a point but towards the end the skipped counter started going and hit what seemed a fairly high number (hundreds of thousands IIRC).

There are some entries that LocalNLP can't use because the don't contain all necessary data. E.g. the last entry in the csv (as of today) is LTE,748,10,29730,1294595,,-53.4549183,-33.6909972,0,1,1,1672764366,1672764366,. Note the empty entry between 1294595 and -53.4549183. This entry is sometimes empty in the MLS file, but required by LocalNLP, and thus the line is skipped. Other reasons for skipping would be if you set an MCC filter to only import data for specific countries.

BTW 1.2.6 from fdroid still crashes with the same error.

It was released before you reported the crash (see https://github.com/Helium314/Local-NLP-Backend/releases), so this is expected.

From what I understand the 'network' provider is actually data provided by the cellular network.

This "network provider" is one of the location providers that should be available on Android. When apps want to know the location, they can request a location from a specific provider. This can typically be GPS, network or fused. Fused is only available if you use Google Play services, GPS should be clear, and network is location information based on networks the device finds. This is usually wifi and phone networks (GSM, UMTS/CDMA, LTE, 5G), and should be added to the system when you install microG.

I'm getting fairly accurate location (dozens of meters) from the MLS backend using only wifi information (airplane mode on) in a reasonable timeframe, we're talking seconds.

This actually is weird. If you have airplane mode on, all radios should be disabled (no wifi, no cell networks, no bluetooth). So MLS backend shouldn't even be able to see wifi information, and even less be able to send a location request to the MLS server. As far as I could find, there isn't even any caching involved in the wifi part...

If you have airplane mode on, and imported only cell tower data, there is no way LocalNLP can find a location. If you disable airplane mode, LocalNLP should be able to get low accuracy location if your device finds any of the cells from the MLS import nearby. Note that MLS file doesn't contain 5G, and sometimes LTE data is incomplete (see above).

If you want more accurate location from LocalNLP, you will need to fill the wifis in the local database first, which essentially happens by having GPS enabled, and getting good accuracy while being in range of a wifi. Then you LocalNLP should report this location whenever your phone detects this wifi.

If you want, you can send logs from the debug version. They contain a lot of data that should allow me to understand why you get no location. But the logs will also contain of information that essentially show your location (nearby wifis and cell networks).

[Mrnofish]

What I mean is that the actual positional information must be supplied by the cellular provider itself. On second thought: I wonder if mobile data needs to be on for that to work? That's usually off for me as I'm on an PAYG plan and mostly use wifi for the internet instead.

I saw a suggestion about switching to a different network provider, but naturally I'd need to buy or borrow another sim to test that. So that's another thing i might try.

FWIW radios work alright in airplane mode. Newer android versions don't even need tweaking to keep radios active when switching it on. Wifi and Bluetooth will stay on if certain conditions are met.

Of course the modem is the one thing that does get turned off, so I set GPS on and airplane off when testing LocalNLP: after setting active mode to high, hours later it was still dead.

I'll see what I can do about the logs. I don't like Mozilla phoning home and I'd rather use LocalNLP instead.

Thanks for being so nice and helpful, wish more devs were this easy to deal with.

[Helium314]

What I mean is that the actual positional information must be supplied by the cellular provider itself

That's not how the network location provider works for any of the backends. What happens is that the backend first checks which mobile cells and wifis are nearby. If it wouldn't crash for you, "Show Nearby Emitters" should show you all nearby emitters (and whether they are in the local database). You can use other apps for that, e.g. SatStat (see the 3rd screenshot).

Once the backend has a list, of nearby emitters, it needs to determine your location from the emitters (wifi and cell towers). MLS backend sends the list of emitters to a server, and the server, which has a database of a lot of emitters worldwide, returns the location data you see. LocalNLP works offline, so it doesn't have the server part. It builds the list of emitters from monitoring GPS locations (see readme), or from importing such a list.

I saw a suggestion about switching to a different network provider

That should not make any difference. Actually both MLS and LocalNLP backends still work if you remove the SIM, because the phone can still find nearby cell towers, even if it's not allowed to connect.

FWIW radios work alright in airplane mode. Newer android versions don't even need tweaking to keep radios active when switching it on.

Ah ok, that isn't the case for my rather old Android 9. But still, you have imported a list of cell towers, but no wifi data. If the modem is of, you will not find the cell towers, which is the only thing you have in the database. So you will need to either build the wifi part of the database before you can get locations, or disable airplane mode.

I'll see what I can do about the logs

From what I understood, I think there is no need for logs currently. If you want to avoid MLS backend, you will need to build up the wifi database first, either by just using GPS a lot, or by switching on active mode (which essentially does the same: use GPS if none of the nearby emitters is in the database).

Another option would be importing a wifi database. I remember there was some place where you could download them per country... Edit: I think that was radiocells.org, but the site seems to be offline

[Mrnofish]

I get the gist of how the NLP backends are supposed to work, I was referring to the network provider that is a different and separate thing from the backends, even though both are referred to as "network location providers" which helps clarity zilch, as the misunderstanding promptly illustrates.

Never mind though, for I bring good news!

Yesterday I reinstalled MicroG as well as the plugins as part of a channel switch, and today every backend is working smoothly.

I'm not quite sure what happened there, but I'll call myself satisfied that I got there. Thanks again for your support!