Only WLAN2 entries in the database.

[DocSniper]

With a new installation without importing anything, there are only WLAN2 entries in the database even after a few days. The exported database then looks like this:

database v4
rfID,rfType,latitude,longitude,radius_ns,radius_ew,note
WLAN2/.....
WLAN2/.....
WLAN2/.....
...
WLAN2/.....
WLAN2/.....

[Helium314]

Since you didn't state what you are missing: Which emitters do you expect to show up? Other WLAN types? Might not show up because your phone isn't capable of using other types, or because you didn't have good enough GPS accuracy. Mobile emitters? They might be disabled in settings, or your phone might provide insufficient data for identification.

[DocSniper]

So, in the Deja Vu plugin I find the following entries GSM / LTE / WCDMA / WLAN2 / WLAN5 and that's exactly what I would expect in this plug-in and no, I haven't deactivated anything, I've activated almost the default setting but "Active Mode" enabled.

Even if I put my phone on the (open) window several times a day with active GPS (in another app), no other entries than WLAN2 and that after weeks of use.

[Helium314]

GSM / LTE / WCDMA / WLAN2 / WLAN5

That's actually what LocalNLP should give, so there seems to be something very wrong. Which version are you using? There were some changes that might affect mobile cell detection in 1.2.2 (though not 5 GHz WiFi...)

[DocSniper]

Oh sorry, forgot to include the version. I'm currently using the F-Droid version 1.2.2-beta.1

[Helium314]

If you use "show nearby emitters" in the settings, you should see all emitters the phone sees and LocalNLP can identify. Does it show anthing else than WLAN2 for you?

[DocSniper]

Yes, the scan shows more than just WLAN2 networks. But often only the message "Scanning failed, maybe the backend service is disabled" appears. Possibly that has something to do with issue #2 Here is a screenshot:

[Helium314]

But often only the message "Scanning failed, maybe the backend service is disabled" appears.

This message appears if LocalNLP isn't running when you start the scan, and fails to start within a few seconds. Start of the plugin is handled by MicroG / UnifiedNLP. Not sure what to do here, but this should not influcence detection of emitters. (maybe MicroG has some limit on how frequently NLPs can be queried)

Screenshot

So there 5 GHz networks detected, but apparently none are saved. This is probably because of GPS accuracy, 7 m or better is required to to store those positions. I assumed every reasonably modern phone would get this accuracy. Do you know whether accuracy could be an issue on your device, or do you usually get better accuracy than the 7 m?

Then there are the missing mobile emitters... Would you be willing to use a debug version, take logs and share them with me? They will contain information about your location.

[DocSniper]

The mobile emitters are sometimes also shown, but the behavior is very strange, sometimes the WiFi emitters are missing. GPS accuracy is not a problem, when I use the phone near the window or outside, the accuracy is often around 4m and Deja Vu registers the emitters correctly. Of course I can test a debug version, should I take it from the release page and which use cases should I reproduce?

[Helium314]

The non-detected emitters compared to DejaVu might be because:

• LocalNLP is using a different way to get mobile emitters, because newer versions of Android might only supply cached (and thus possibly outdated) data
• LocalNLP discards WiFi scan results that are not updated (same as the previous scan) Some Android versions limit how many scans can be done in a certain period, and only return old data within this period. Android 9 is especially bad, because there it normally can't be turned off.

I implemented this because I noticed I often got wrong positions caused by old / cached scan results (which are used by DejaVu).

The debug logs contain the full data, so I hope to see what has been discarded and why. Ideally I would see in the logs when you have GPS active, to see which emitters are stored and which are not.

You can use the debug version available on the release page, but this will clash with the F-Droid version you have. If you don't want to uninstall it, use the version attached, which can be installed together with the normal version. (this will be default in the next update) local-nlp-backend_1.2.2-beta.1-debug.zip

[Helium314]

Do you know how to get the logs (logcat)?

[DocSniper]

Thanks for your detailed explanation. I know my stuff well and will get the logcat. I hope to have a log for you by tomorrow.

[DocSniper]

Here's a quick log, I hope that's enough, if not I'm happy to do more. Edit: I hope the notification mail was sent, so the link here has been deleted.

[Helium314]

Thanks, got it. Not sure if I find the time for a closer look today.

But a quick look: emitters are detected correctly, but LTE is not inserted becuase of some issues with the timestamp. Emitters are discarded if timestamps of GPS location and scan result differ too much, because I sometimes got several minutes old scan results that Android reported as new. In your case it looks like the timestamp is different from the one my phone uses. This might be a difference between realtime and uptime, as it is not specified which one is used in the [documentation](https://developer.android.com/reference/android/telephony/CellInfo#getTimeStamp()).

Maybe the fix is simply using the new getTimestampMillis method for Android 11+... which version are you using?

[DocSniper]

I'm using LineageOS 18.1 which is Android 11. Take your time, don't hurry. Good night. 😉

[Helium314]

The issue why WiFis often are not stored is that your phone seems to throttle WiFi scans, and GPS locations only come from active mode. The typical flow (with active mode) is:

• network location is requested
• phone scans for emitters
• emitters are found, but none is in the database, so GPS is started (when active mode is enabled)
• emitters have been found, but no GPS location -> nothing to update
• after some time, GPS location is found, which triggers a new scan for emitters
• but phone doesn't actually scan for emitters (reports the scan as not successful), most likely because the previous scan wasn't long ago
• no scan -> no emitters -> nothing to update

I adjusted the flow, so the emitters found before enabling GPS are used too (if they are not too old).

Would you do another test with the updated version? This should also have the issue with the timestamps fixed, so LTE cells should now be stored. local-nlp-backend_1.2.2-beta.1-debug.zip

[DocSniper]

After a quick test, I have now seen WLAN2 and LTE entries, but no WLAN5 entries yet.

[DocSniper]

Was cycling today and recorded two logs. One in active mode and one in passive mode. Seems to work quite well now, but the WLAN5 entries are still missing. Edit: As before, I hope the notification mail was sent, so the link here has been deleted.

[Helium314]

Thanks, I got the logs. Tough no time to check them... may take a few days.

[Helium314]

The logs are a bit short, but it looks like there is nothing obviously wrong here. WLAN5 is not stored because the best accuracy in the log is 8 m, with is above the 7 m limit for WLAN5. In the previous log there were some GPS locations with better accuracy, where WLAN5 should be stored with the changes done in the second test version.

Actually WLAN5 often is not stored when using active mode, because active mode stops GPS once sufficient accuracy for WLAN2 is reached. This is done to avoid excessive battery drain when GPS signal is bad. So for storing WLAN5 it's better to get GPS from some other app, e.g. having some map or GPS test tool running (essentially like in DejaVu).

[Helium314]

On a second look the active mode log is rather strange... normally a message with getGpsPosition must have onReceive() - received intent just before, because getGpsPosition is only called from within onReceive. But this is not the case in the new log.

Did you do anything different than previously? The old log was fine in that respect, and the relevant code was not changed between the two versions....

[DocSniper]

I have now tested the passive mode and yes you are right, with GPS more accurate than 8 m WLAN5 entries are also made, so far everything seems to work.

When recording the logs I did not change anything from before, I only had the switch active mode once on and once off when recording.

[Helium314]

Good to hear it works. Though I still wonder about the logs... not even aggressive power saving should be abe to do this.

Anyway, a new version with the fix should be available on F-Droid soon.

Did you intend to fill up the database using active mode only? I.e. without relying on other apps using GPS? I'm considering adding an optional more aggressive active mode that would help, but it turned out to be a bit more work than it appeared initially...

[DocSniper]

I used active mode because I wanted the database to be up to date as quickly as possible for where I'm most often. After a few days I would have switched back to passive mode. The same, since I rarely use GPS, so it shouldn't be active too often.

But if you want, I can create a log again. If so, do you want active or passive or both?

[Helium314]

No log is necessary.

Maybe for your use case some more aggressive active mose would be useful. The main "issue" is that active mode starts GPS only if no position could be determined. Since low accuracy position (e.g. some GSM cell) is enough, active mode might only start rarely (bad for precision / finding WiFis, but good for battery).

[DocSniper]

No problem, I will now stay in passive mode and then activate GPS with the help of another app if I want to have a certain location in the database for sure. But of course you're right, an even more aggressive mode could certainly help here too.

[Helium314]

I extended the active mode setting.# If you set it to medium or high it should be more useful for filling up the database now, without using too much battery.