Autosplit the current session

[sim6]

Hello,

Fisrtly, thanks for your work.

When Radiobeacon Android client has scanned hundreds of WiFi cells during the current session it becomes slowly and eats my battery. Please, could you add a feature to split the current session automatically?

Thanks again.

[wish7code]

Hey Simó,

short update on this one: I assume this happens in map display mode, right? I analyzed the code and found some potential memory leaks..

As soon as I get the remaining issues in the nightly builds fixed, I'll update F-Droid release..

Cheers Toby

[sim6]

No, I don't use the map. I start a new session and the overview is displayed, then I lock the phone. If it scans more than 500 wifis then when I want to restoring the application takes too long to display the overview again.

[mvglasow]

Same thing here. It is easiest to observe in map display mode, as updates take really long (and it also seems to have an impact on measurements). However, I've also observed it in overview screen when tracking with two devices, one of which had a fresh session while the other one had been tracking for a while.

Today I had the issue again. I stopped tracking and restarted, to no avail. Then, I killed the app (with a long press of the Back key), restarted it and resumed the session, and things slowed down again within a few seconds. That means it is really the size of the current session that matters, not the time spent logging.

About autosplit – if you decide to take that path, there's the map display issue. I rely heavily on the map (to check where I've already been), thus we need a way to ensure the first part of the session doesn't suddenly disappear from the map when the session is split.

[wish7code]

Since it happens not only in map mode, as Simó observed, I was wondering whether this may be related to the 'not-so-smart' way we currently handle status bar updates :-(

We've got some potentially heavy database operations to count current session's number of wifis @ https://github.com/wish7code/openbmap/blob/master/android/app/src/main/java/org/openbmap/activities/StatusBar.java#L153 . This line is called on each and every observed wifi and opens a database cursor @ https://github.com/wish7code/openbmap/blob/master/android/app/src/main/java/org/openbmap/db/DataHelper.java#L176

We might simply put to much pressure on the database by doing this...

Instead I think we should cache the number of wifis in the activity and remove the database access at this point

[mvglasow]

That should be fairly easy to detect – call System.nanoTime() before and after the operation to measure duration. If it's the DB operations, we should see the duration for these queries start out small, then grow in duration as the session gets larger.

As for the counter – the main challenge I see is deduplication. Assuming you have just picked up BSSID 00:0b:ad:c0:ff:ee, then move out of its range and at a later point pick it up a second time: you want to make sure we don't count it again. For the count of new wifis, this is even more important.

Therefore, we'll probably still need to do two database queries for each wifi in a scan result:

• Has the BSSID already been seen in the current session? If not, increment wifi count.
• Is the BSSID already in the wifi catalog? If not (and it's the first time we're seeing this BSSID in this session), increment new wifi count.

Without having looked through the code in detail, the second query is probably already there, and it shouldn't cause any performance issues – if it did, they'd appear right at the start of each session. The first query could be modeled after it. Rather than selecting all matching rows, you might want to use EXISTS, which is potentially faster (the DBMS can stop searching after the first hit).

I had a look at indices: I see there is already an index on wifis(session_id) and on wifis(bssid, level) (the latter of which will also work for lookups by bssid).

Running

explain query plan select * from wifis where bssid='00:0b:ad:c0:ff:ee' and session_id = 42;

returns

SEARCH TABLE wifis USING INDEX idx_wifis_sessions_id (session_id=?)

which means the query walks through all the wifis in the current session. I'd suggest extending the index on wifis(session_id) to wifis(session_id, bssid): it would still work for lookups by session_id only (session_id being the first column in the index) but also speed up queries like the above.

[mvglasow]

I've extended the index as described in my local Radiobeacon db and will see how that affects behavior. After my next longer log, I'll report back.

[mvglasow]

I went on a longer logging trip today and it looks as if the index alone doesn't help – at around 3000 wifis map updates slowed down noticeably. After I started a new session, everything was fine again.

[wish7code]

I know this is a long time issue, but by instance I discovered something really strange today:

I started RadioBeacon without no local map installed. In that case RadioBeacon falls back to using online maps.

Right now I've got a session with ~5000 wifis and no observable lag. So it might be a bug related to mapsforge offline maps?

Coud anybody verify this behaviour?

[mvglasow]

RadioBeacon falls back to using online maps

Interesting... didn't know that was possible. Is there a way I can choose between online and offline maps? (If not, that would be a feature request... most of the functionality is probably there already.)

If it really is mapsforge (and only when maps are rendered locally), I do find it somewhat surprising that this also appears to happen when overview is active (afaik Android keeps only the active page and its immediate neighbors around, thus the map page wouldn't even get created until I switch to its imediate neighbor).

I'll try one thing when I have the chance to: I'll disable map centering so that position changes will not trigger rendering operations, and then see what happens.

If it really turns out to be a mapsforge issue, the guys there appreciate it when you can give them steps to reproduce the behavior with their sample app, though that might be difficult here. Maybe plugging into Mapsforge's test harness is the way to go here: request a handful of tiles to be rendered over and over again, and watch rendering times – they should not change noticeably.

[wish7code]

If not, that would be a feature request

Makes perfect sense, Greg already open a feature request, see #153

afaik Android keeps only the active page and its immediate neighbors around

Correct, but unfortunately it's seems to be impossible to deactivate that caching completely. To make it worse, I think I remember that sometimes Android gets confused on what is the immediate neighbor to load after switching tabs. So the map component might be loaded even though it's not a immediate neighbor.

[mvglasow]

I recently learned through painful experience that certain Mapsforge components require cleanup when they are no longer needed – not doing so will eventually exhaust all available memory – I haven't studied the exact behavior, but since an app with a memory leak would require more and more memory, the system may need to do stuff in the background (kill apps in the background, start garbage collections etc.) which would consume processing power, slowing it down.

In the onDestroy() method of your Activity or Fragment, call the following:

• onDestroy() for every layer in the map view
• destroyAll() for the map view
• destroy() for every tile cache
• AndroidGraphicFactory.clearResourceMemoryCache()

More important, every time you remove a layer from the map view, you need to call onDestroy() on it. IIRC, Radiobeacon creates layers dynamically for the overlays – could that be the issue? Maybe watching memory usage of the app over time would help.

[mvglasow]

I upgraded from 0.8.15 to 0.8.18 last week, and the new version exhibits this behavior after just a few minutes of tracking.

I have tested this in two setups:

• Observing map display
• Direct comparison between a Nexus S running 0.8.15 and a OnePlus One running 0.8.18. Even without ever switching to the map view (both devices stayed on the Overview page), the OnePlus One soon starts to get sluggish, with WiFis appearing several seconds later than they did on the Nexus S. Note: Wireless scan mode is set to Background scanning.

Which leads me to the two suspicions:

• It's not the map view but something else.
• Something which changed from 0.8.15 to 0.8.18 made matters even worse – though I have no idea if what this could be. Maybe measuring the duration of certain operations (using System.nanoTime() as described before) would provide a clue...