Tracks on display slow Osmand down (configure map > tracks is terribly slow too)

[Rowin63]

Description

One thing is, since some time (some weeks) Osmand slows down terribly as soon as there are some tracks on display. Getting worse by every displayed track. The other thing is, the tracks section ("configure maps > tracks") takes about 1m20s until the content is displayed (this does not happen via "my places > tracks"!)

Steps to reproduce

I don´t really know, but it obviously is caused by the number of stored/displayed tracks. Without tracks the app "configure map > tracks" runs smooth. Without tracks on display the app itself (zoom/pan the map) runs smooth.

Actual result

I did several fresh installs the last days and tried different situations (changing number of maps/favorites/tracks/contour lines/wiki/overlay map/... contained in the apps folders). A) As long as there are no tracks on display, the app runs smooth. As soon as I swith the first few traclks on display, Osmand slows down.

B) As long as there are no tracks stored in the app, "configure map > tracks" opens quick As soon as I store tracks in the app, "configure map > tracks" gets slower and slower by the number of tracks

The only difference I see: this happens only on my Samsung Tab S6 running Android 12 (since some weeks) but not on my two other devices running Android 13 (Tab Act3, S20+).

I use androids "media"-folder, but this problem also exists in "data" ("extern 1")

But anyway it´s a "new" behaviour and did not happen before. Data is the same on all devices and far away from the previous "oom"-numbers.

Expected result

Don´t slow down like this...

Your Environment (required)

OsmAnd Version: +4.7.17 Android/iOS version: 12 (Tab S6), 13 (Tab Act, S20+)

[yuriiurshuliak]

The bug has been reproduced. Initially, zooming and scrolling the map without displaying the tracks worked smoothly and consistently. However, once I enabled the display of 47 tracks, the map began to slow down and lag when zooming and moving around.

OsmAnd~ 4.8.0#2476m, released: 2024-06-10

No tracks on the map	With tracks on the map

[Rowin63]

Thanks for investigating and confirming 👍 Please tell me, do you see this related to Android 12?

[yuriiurshuliak]

It's unclear if the issue is related to the Android version, but I'm using Android 12.

[sonora]

In my experience, it is a weakness of the v2 engine. I am using Menu > Settings > OsmAnd settings > Map rendering engine > Version 1, with much better performance (Android 11). May depend on device, though.

[Rowin63]

I already switched back to V1, does not make a difference in this issue. The main problem is this looong time Osmand needs to display the tracks in "configure map > tracks". As mentionend above, it takes much more than one minute until the tracks are processed/displayed. But now I see, in fact this also happens somehow in "my places > tracks". Here at least the folders are displayed almost immediately (this first led me to think it does not happen here), but also here it takes a lot of time until all the tracks are processed and finally listed.

Strange thing is, this does not happen at all on my other (android 13) devices. Everything is displayed within a few seconds. And the Tab S6 should be better hardware than the Tab Act3 is. Though it does not happen on the Act3. So the only dfference I can see is A12 and A13. It´s all the same amount of data on all devices (I´m synching regulary). Dont know what else to do...

And I did really clean/fresh installs (deleting all remaining folders/data via file manager after de-installing...) Unfortunately I can´t name a time, when this problem began. It was a few weeks ago.

[sonora]

Which build number exactly? I had a similar issue and have done a lot of work in the last 3 weeks or so on that, finally bringing things down to 1-3 sec for either screen (for my setup with~4000 tracks on 100 folders). But it depends a bit on the exact build, because we had some interim commits which had brought back the "slowness"... (and I think the public release is still missing this optimization entirely.)

In any case I think optimizing the code for these two screens has now reached its limit. There are 2 other epics on the way optimizing the Track loading and caching in general, they are still work in progress.

[Rowin63]

I have to reduce the content of the tracks folder to a REAL minimum (120 tracks, less than 50Mb), then the app runs smooth and the tracks list is displayed within a few seconds. No matter if 5 or 50 maps downloaded, it's absolutely depending on the amount of stored tracks.

[Rowin63]

It's the +4.7.17 from playstore.

But again, as mentioned before, it only happens on the Android 12 device. Don't have this issue at all on the other Android 13 devices, they're running smooth with >400Mb of stored tracks.

[sonora]

The things I fixed was avoiding repeated file I/O, and removing recursive code. While the first impacts most devices, the second point depends strongly on how much memory a device has and how it allocates/reuses it during excessive recursion. So quite possible that you observe such difference between devices, and also quite possible that if you test a nightly build or wait for the next release, you could see a decisive improvement. :wink: I for my setup saw an overall improvement of a factor of 10-15...

EDIT: The above is merely for entering the MyPlaces > Tracks and Configure maps > Tracks screens. Not for map screen lag, not sure which factors bottleneck that....

[Rowin63]

Switched over to 4.8.0#2491. Same issue (loading/processing/displaying tracks is only a bit quicker).

) No (or just a very few) tracks stored: "my places/tracks" and "configure map/tracks" works quick. ) The more tracks I store in the app (in steps up to 800 tracks / 250Mb > that amount never was a problem the last months) the slower the processing is (track list in "configure map / tracks" still takes >30s to be displayed)

No tracks on display: map pan&zoom is smooth as usual. Around 50 tracks on display: smooth pan&zoom is not possible.

In sum it´s a bit better now with the nightly than it was with the last playstore release. But still bat compared to a few weeks ago. Mentally I´m still stuck to the A12. This is the only obvious difference betrween all my devices to me. I didnt do anything experimental things on this Tab S6 (like root or things like this), just very normal use.

I´m at the end of my ideas now... Is it caused by the Tab S6 (hardware) itself? (if so, why hasn´t it been like this always...?) Is it caused by Android 12? Is it... I don´t know?

But wasn´t there a fix to "remember" the content of the track list? I see it is re-processed everytime I leave it and go back to it again...?

[Rowin63]

Just to countercheck this issue. I organized another Tab S6 (same SM-T865, same Android 12), installed the same nightly and it shows the same issue with readig/processing the tracks. So it can be caused by the Tab S6 itself (but it's an Octacore, 8Gb/256Gb) , or the Android 12. I'm still confused, not an attempt of this issue on my (less powerful) Android 13-devices...

[sonora]

Does it make a performance difference if you use external or internal storage for the OsmAnd data folder?

[Rowin63]

Have to check this. I'm using the "media"-path (for access/synching reasons), and I've tried the "obb" -path, no difference.

Next I'll give "internal" a try.

[Rowin63]

Does it make a performance difference if you use external or internal storage for the OsmAnd data folder?

Pfffthhh... Impressive. Using the "internal" path changes Osmand into a rocket... Tracks/content ist displayed/processed in the fraction of a second. I can't even notice a delay...

This shows this issues cause, but I can't use the internal memory, it's hidden and I don't have access to my data per file manager...

And the cause must be hidden in A12 (there was an update recently, but I can't remember if this triggered this issue) or in Osmand.

[sonora]

Ok, that confirms my suspicion that file I/O is causing the issue here, like often in OsmAnnd. And with that it in deed may be a peculiarity of rhe particular hardware and/ or Android 12. And we are rather deep in google's 'secrets' of modifying file access mechanisms over the years. 😞

Chances are that a build with 'All files access' permission would solve the issue (,#16059) which would allow using any path (like just sd-card/osmand) as the data folder, and Android would then usd a different I/O access mechsnism, I think.

There may be other workarounds like better chaching, or using transparent zipping (#10653).

[sonora]

PS: Perhaps using the Download folder (or on some devices subfolder of it) as OsmAnd's data folder makes a difference, not sure. If yes, it would mean that the media access I/O on some Android devices is just slow (and probably noone cares, beceause for listening to a song or watching a video it's still good enough...)

There are also marginal chances of cheap SD cards limiting this, or a device having a slow sd-card reader. But foremost I suspect it's the way Android 12 handles file I/O, and/or perhaps media access.

EDIT: In deed, upon some research: From https://developer.android.com/training/data-storage/shared/media and elsewhere: In Android 12, when you perform random reads and writes of media files using direct file paths, the process can be up to twice as slow. There are also reports on Samsung forums about performance degradation following an Android 12 update, some say wiping the cache partition has helped. But I suspect it would be good if in OsmAnd we tried some of the 3 mitigations listed in my prior post.

[Rowin63]

I will move the data folder to "downloads/...) and give it a try. But meanwhile I'm getting closer and closer to the last A12 update (and your research pretty confirms that)...

[Rowin63]

Moved the data folder to download/net.osmand.plus/files Yes, it's faster than the media path is (ca 15sec to display the track content), but zoom&pan still is... no fun.

[Rowin63]

Also "data" is (and "obb" probably will be too) quick. But unfortunately no synching (anymore) possible in this paths...

[sonora]

Yes, my understanding is google also uses yet another file I/O mechanism for the Download folder, amd also that is severely (performance)-limited vs. what the hardware could actualy deliver.

The only sure-fire way to mitigate this reliably seems to use an OsmAnd build which simply gives the app the 'All files' access. Something google wants to restrict to file manager type apps ever since Android 11. Hence my attempt to convince Victor registering OsmAnd as a GPX File Manager app (#16059)...

If you run your own private build, it's a 1 line change in the code to try...

[Rowin63]

OsmAnd as a GPX File Manager app (#16059)...

I know there's a discussion about this...

If you run your own private build, it's a 1 line change in the code to try...

Unfortunately not. I don't have any knowledge about that..

But thanks anyway for your help 👍

[sonora]

Perhaps we could convince Victor to publish a 'test' 😉 OsmAnd build with 'All files permission' on the OsmAnd download server... but I am afraid no one may then use the official 'google' build any more ... haha... 😉😘😉

[Rowin63]

I would like to beta use it 😉👍

[Rowin63]

I want to give you an update, that solves my problem (extremely slow access to "media" folder in Android 12) for the moment (I had to move Osmands data to "media" , because I couldn't synch anymore the "data" and "obb" path):

For some reason it is possible again, to add "obb" (but not "data") as a "user defined exception" in the synching app I like to use (Folder Sync Pro). So now I am able again to use the (quick) "obb"-(multiuser 1)-path as Osmands Data Folder, though having access to Osmands data for synching 👍

[sonora]

Thanks! I will add this hint to the documentation https://osmand.net/docs/user/troubleshooting/maps-data/#maps-slowly-loading-on-android-11-12-sd-card. Do you use 'Manually specified' or one of the other options in OsmAnd's Data folder menu? Multiuser storage 2, I assume?

PS: I just see this issue reported about using the obb folder https://osmand.net/docs/user/troubleshooting/maps-data/#deleting-map-data-after-the-app-update-if-selected-multiuser-storage-1 in our documentation there. Hopefully this is a 'ghost of the past' not sure that issue is still around...(?)

[Rowin63]

I use Osmands predefined "multiuser 1" (the internal memory, "2" would be SD card, never tried that).

[Rowin63]

The mentioned issue about deleting maps at an update did not happen anymore (to me 😉), did some updates yesterday

[sonora]

Ah ok! But then it's simply fast because you use the internal storage... So it's no real solution for users who have limited internal storage, and are looking to use an external SD card's folder readily accessible by other processes, e.g. for syncing...(?)

[Rowin63]

Ah ok! But then it's simply fast because you use the internal storage... So it's no real solution for users who have limited internal storage, and are looking to use an external SD card's folder readily accessible by other processes, e.g. for syncing...(?)

I will give it a try later today. Just need to move the app over to "2". Will tell you then 😉

[Rowin63]

Moved the data folder to "multiuser 2" (obb/SD, used Osmands predefined setting). Is as slow as media path. Tracks menu content takes a minute to show up, zoom&pan (but in fact everything you do in the app...) with tracks on display is unusable slow. I'm happy that internal obb at least runs fine 👍

[sonora]

Yeah, expected that. It's Android new mechanisms to limit SD card access to the protected (hidden) file paths, or to certain library mechanisms like Media or Downloads, which uses slow processes.

So the 'All files' permission remains the only known mechanism so far to facilitate using a 'generally visible' folder on the external SD card with the max performance the hardware can deliver.

[vshcherb]

@sonora we are publishing your build already, and we are publishing android flavor which is different than google play and some features might be present in Android flavor for example