osmandapp / OsmAnd

OsmAnd
https://osmand.net
Other
4.4k stars 989 forks source link

Slow Map Loading on SDcard #13943

Open Ryu945 opened 2 years ago

Ryu945 commented 2 years ago

Description

Slow Map Loading on SDcard

How to reproduce?

Have the map database downloaded to SD card. Then open the app when it is fully closed.

Your Environment

OsmAnd Version: 4.1.11 (Fdroid Store) Android version: 11 Device model: LG V20 910

Maps used (online or offline):
Any offline map

Whenever I first start up the app, it takes several minutes before I can use the app. A friend of mine who has the exact same phone but loaded the map on the phone's internal storage does not have this problem. This is the memory card: https://www.bestbuy.com/site/samsung-pro-plus-512gb-microsdxc-uhs-i-memory-card-with-adapter/6473932.p?skuId=6473932

I don't know if this is a bug that is slowing down the access of data on the SD card, a bug that just makes data access slow regardless of where you store it or if the map app just requires ridiculous levels of bandwidth to storage to operate.

kevinmcmurtrie commented 2 years ago

Google Play forces apps to use crippled microSD access APIs. OsmAnd could work around this if they modified their F-Droid version.

scaidermern commented 2 years ago

Related to #12046? @Ryu945 This issue mentions a workaround.

kevinmcmurtrie commented 2 years ago

The workaround requires OsmAnd~ 4.0.9 from F-Droid. You can search for the word "slow" in issues.

I may try a custom build with MANAGE_EXTERNAL_STORAGE set but I figure the prep time is pretty much a whole day. The Android tools on my computer haven't been used in years.

scaidermern commented 2 years ago

As I understand, the workaround is to use a different storage folder. This doesn't require the f-droid version or a custom build.

Try setting the storage location to /storage/XXXX-XXXX/Download/osmand where XXXX-XXXX is the name of your SD-Card.

kevinmcmurtrie commented 2 years ago

On my Sony and LG, the Download folder is slow and it doesn't have read-after-write consistency. OsmAnd will lock up a lot and crash with I/O errors. It might work on a Samsung that tends to be less stock Android.

It's amazing that Google has failed so badly that Samsung OS patches are a good thing.

Ryu945 commented 2 years ago

On my Sony and LG, the Download folder is slow and it doesn't have read-after-write consistency. OsmAnd will lock up a lot and crash with I/O errors. It might work on a Samsung that tends to be less stock Android.

It's amazing that Google has failed so badly that Samsung OS patches are a good thing.

I am trying to figure out what folder location I can use on the external memory of an LG phone that doesn't go slow and has anything important it might need like read-after-write consistency.

kevinmcmurtrie commented 2 years ago

I tried to make a custom build but couldn't figure out what was going wrong with the Gradle files. The old version from F-Droid is the only way I can use it.

Ryu945 commented 2 years ago

I tried on an LG V20 to change the location of the folder to several other locations on the SD card and it is still going very slow.

Ryu945 commented 2 years ago

I haven't tested this but I am curious if anyone knows if this resolves the problem. It is possible on Android to tell it to add the SD card as part of the phones storage. Will map data stored in the SD card in this way not have to go through the API that makes data transfer inefficient?

kevinmcmurtrie commented 2 years ago

I haven't tested this but I am curious if anyone knows if this resolves the problem. It is possible on Android to tell it to add the SD card as part of the phones storage. Will map data stored in the SD card in this way not have to go through the API that makes data transfer inefficient?

I tried using ADB commands for format a microSD card as internal storage on a phone that didn't offer it. It was fast but the user-level mount point was missing. Apps could read and write from their scoped area just fine but there was no browsing or sharing files across apps. All file utilities, including the ones built-in, were broken.

tom-ch1 commented 2 years ago

Is it possible to bind-mount on Android? Would permissions and speed be solved by this? I am thinking of something like this on startup: mount -o bind /storage/1234-ABCD/OsmAnd /storage/emulated/0/Android/data/net.osmand.plus

kevinmcmurtrie commented 1 year ago

As mentioned in another thread, the latest F-Droid release lets you manually enable the filesystem management permission (system App Info setting). This restores the ability to set a fast microSD path.

Ryu945 commented 1 year ago

As mentioned in another thread, the latest F-Droid release lets you manually enable the filesystem management permission (system App Info setting). This restores the ability to set a fast microSD path.

Where? I tried updating the app and reinstalling the maps. It is still the slow API.

sonora commented 1 year ago

Grant " Storage / All files access" in "Android settings / Apps / OsmAnd / Permissions", then chose a custom path like "extSdCard/osmand".

kevinmcmurtrie commented 1 year ago

And don't use any Google directories (Alarms, Android, Documents, Download, ...). They may be restricted to the slow file APIs. Select or create an OsmAnd directory at the top level of the microSD storage in the app.

Ryu945 commented 1 year ago

Grant " Storage / All files access" in "Android settings / Apps / OsmAnd / Permissions", then chose a custom path like "extSdCard/osmand".

The permission list for OsmAnd does not mention that permission. It only has "allow access to media only"

I looked in special access permissions and found access all files. OsmAnd is not listed as an app capable of turning that on.

kevinmcmurtrie commented 1 year ago

Make sure you're still on OsmAnd~ 4.2.6 or later from F-Droid. I've twice had Google Play overwrite it with OsmAnd+ without permission.

Ryu945 commented 1 year ago

Make sure you're still on OsmAnd~ 4.2.6 or later from F-Droid. I've twice had Google Play overwrite it with OsmAnd+ without permission.

The latest version out in the Fdroid Store is 4.1.11

Ryu945 commented 1 year ago

Make sure you're still on OsmAnd~ 4.2.6 or later from F-Droid. I've twice had Google Play overwrite it with OsmAnd+ without permission.

I looked into the version details of OsmAnd versions. Fdroid store does not recommend any version past 4.1.11 so apps will never update past that. I had to manually click to update to the 4.2.6 version.

Ryu945 commented 1 year ago

From what i can tell. Once I forced it to update to 4.2.6. Then I changed the file access permission. Then I changed the directory to a non android directory. I finally saw an increase in speed assuming it is not the placebo effect.

kevinmcmurtrie commented 1 year ago

Not sure if this is still an active bug, but OpenGL rendering has some issues when not using SAF.

Get the F-Droid version of OsmAnd~ and run appops set --uid net.osmand.plus MANAGE_EXTERNAL_STORAGE allow in an ADB shell. Set a custom storage location (not SAF) on a microSD card and use OpenGL rendering.

On launch, the app renders nothing but an empty grid while Google's media services is 130% I/O wait. It stays blank for anywhere between 1 second and longer than I'm willing to watch the phone. The performance is normal once it finally starts working but OsmAnd is logging crazy amounts of errors about truncated OBF files. The length it's reporting for the file seems random. The files are actually fine.

endolith commented 2 weeks ago

Is this still a problem in Android 14? SD card storage seems slow but I'm not sure if that's because of this or just because it's slow.

kevinmcmurtrie commented 2 weeks ago

Is this still a problem in Android 14? SD card storage seems slow but I'm not sure if that's because of this or just because it's slow.

I think the "solution" is that Google has broken microSD cards so badly that phone makers don't include them anymore. SAF is still entirely unusable today - it cripples shared file performance even on motherboard storage.

sonora commented 2 weeks ago

Yes, exactly. the hardware can do a lot better, it's the software limitations in the OS policies and mechanisms creating the bottlenecks here.