Maplibre - Githubissues

Helium314 commented 4 months ago

For simple comparison with master, and for discussions.

to do

~quest pins: replace homebrew solution for quests and overlays with CustomGeometrySource (draft started by @Helium314)~ blocked by at least maplibre-native#2262 and not strictly necessary

to do SCEE

re-implement aerial imagery style: style json, adding and changing aerial imagery layer
(optional:) if quest pin size has considerable impact on performance with MapLibre, a SCEE setting to half the size of quest pins would be conceivable (maybe some sort of ancient-phone-mode)

upstream blockers

minor: maplibre-native#2255 - annoying, but of little consequence
~cosmetic: maplibre#2259~ - worked around this issue
crash on adreno 3xx GPUs: maplibre-native#2206
critical: maplibre#2410 - users MUST see all available data for the places and things overlays to work as intended. Unknown how many devices it affects and/or how often this occurs for these.

old observations below

Performance regarding icons (symbol layers, mostly about quest pins):

Mostly fine at the current state, comparable to Tangram
Choppy zoom, especially if there are many quests displayed
- zooming during high CPU load (on UI thread) makes it much worse, e.g. when opening a quest
- on A3 it's not that bad, would like to test the current state on S4 mini...
Image (quest pin) size has a considerable effect on performance
Clustering improves performance
Using some sort of ordering for the symbols from a property is (noticeably) slow. Currently the order is set to the order of adding symbols to the source. This requires a sorted list of quest pins, but still it's fast enough. No need for a pre-quest order like in Tangram.
- Sorted list is not working with clustering
- does not seem slow on A3, would like to test the current state on S4 mini...

Other things:

MapLibre always does some downloads right after start. Possibly glyphs, considering that when you start SC without WiFi there is a message This request was cancelled (https://api.jawg.io/glyphs/Roboto%20Regular%2cNoto%20Regular/0-255.pbf). This is expected for tiles that were being prefetched but are no longer needed for the map to render.. It's not horrible, but downloading the same thing on every start seems like a waste of bandwidth and resources.
- maybe goes away when properly pre-loading tiles for offline use

westnordost commented 3 months ago

Uh, oh, that's right, I didn't notice that it is even inconsistent within the app. For markers, the text is below, for overlays, to the right. But I think I know the reason why I did that and it doesn't apply for MapLibre anymore, so I can remove that. (Because the left-alignment also causes issues for the address overlay)

westnordost commented 3 months ago

I'll be looking into the location component

westnordost commented 3 months ago

I pushed the changes into a branch... overall, I am not that convinced that we should use it. Reasons:

It is not well documented. There are tons of options and very few actually helpful explanation what what does
Some things don't seem to work, e.g. the bearing display
(Animated) movements are very jerky, some other things noted in the commit message
This is completely an Android API, i.e. on the native level, it uses the GeoJson sources. iOS and Android SDKs are very different in this aspect, it seems. For iOS, there is just a single bool to show it or not. Should iOS and Android thingies work the same, the code must be in the app

westnordost commented 3 months ago

(not done with it. I will try to get smooth animation of position + maybe even accuracy circle working. Then, I'll consider moving the "follow location" and "navigation mode" stuff into that component.)

Helium314 commented 3 months ago

I tried the LocationComponent a few times, but the accuracy circle behavior is really weird. Smooth animations for the current (non-MapLibre) style is all that is missing I think.

(I finally had a little time to build the maplibre test app, hopefully it will work on my phone at some point)

Helium314 commented 3 months ago

@westnordost I'll be able to test on a 64 bit capable phone starting tomorrow, so I can finally continue here... (any I'll have more time than in the past 2 weeks)

westnordost commented 3 months ago

Nice!

(Which phone? Are you planning to change your EDC phone?)

Note that (in regards to the description text of this PR) quite a few things, especially visual ones, have been solved or cleared up. We won't use the compass, attribution from uiSettings and we won't use the locationCompoment.

Helium314 commented 3 months ago

Note that (in regards to the description text of this PR) quite a few things, especially visual ones, have been solved or cleared up. We won't use the compass, attribution from uiSettings and we won't use the locationCompoment.

I'll check and update the initial post (if you want you can also edit / add / remove stuff).

Which phone? Are you planning to change your EDC phone?

I got a Galaxy A3 (2017), which is the smallest Android phone with OLED display that's clearly better than my S4 Mini VE. If I find it acceptable I might actually use it as EDC, not sure yet.

Helium314 commented 3 months ago

Good thing: it finally works (no crash) Not so good thing: zooming out with quests displayed is still somewhat janky, with tangram it's smoother even on the original S4 Mini Other not so good thing: quest icons and pins are misaligned in second tutorial screen

westnordost commented 3 months ago

Well, there are two things that will likely make it smoother: quest clustering + removal of quest dots and CustomGeometrySource you have been working on.

Because:

removal of quest dots cuts number of displayed quests in half
introduction of quest clustering further reduces the number of displayed quests.
I believe the manual re-setting of the whole GeoJsonSource every time a new area comes into view (e.g. on zooming out) is somewhat wasteful. I hope / think that CustomGeometrySource only adds the data from a tile not added yet

westnordost commented 3 months ago

Galaxy A3 looks still reasonably small. The next more modern phones when OLED is a must would already be my Samsung s10e (2019) and the Sony Xperias with the stupid naming scheme, both quite a bit larger.

Anyway, next thing I intend to do on this PR is to maybe move the follow-position and navigation mode stuff also in the location component (to mirror the capabilities of the locationComponent). If not, then make it an own component. And then, consider merging the MapFragment, LocationAwareMapFragment and MainMapFragment into one. My original reason for this separation was 1. keep the files small, 2. maybe there will be a second map somewhere in the app that should e.g. not display the quests but other things (e.g. download dialog, splitting way etc..). Reason 1 should rather be solved with utility / extension classes and functions (like these *Components) plus viewmodels and reason 2 never came to pass and I find it unrealistic that it would ever. A strong reason for merging the three is https://github.com/Helium314/SCEE/pull/516#issuecomment-2008437828

I will likely continue next week.

Do you intend to continue soon? I am currently a digging through all fragments and restructuring them with ViewModels, so I can imagine that I am generating quite some (at least: merging) work for you. I am not changing anything closely related to the map, of course, taking into account this PR.

Helium314 commented 3 months ago

Galaxy A3 looks still reasonably small. The next more modern phones when OLED is a must would already be my Samsung s10e (2019) and the Sony Xperias with the stupid naming scheme, both quite a bit larger.

Still not sure whether it will be EDC, it feels too large and slippery (the rough plastic back on the black S4 Mini is really nice).

Do you intend to continue soon?

I don't have much time, and currently learning a lot of new stuff on my job which can be a little exhausting... I'll try working on polishing the offline stuff, which is mostly MapTilesDownloader. So I don't think there will be many merge conflicts here.

Btw should be merge the current master? Or do it only in the end?

westnordost commented 3 months ago

The slippery glass back of my s10e also bothers me, ultimately I bought a silicon casing because of this.

Oh yeah, that can be exhausting.

Btw should be merge the current master? Or do it only in the end?

I think that makes sense. I was also thinking whether it would be better to rebase this PR on top of upstream master, not sure how well a merge from SCEE to StreetComplete would work, I don't have any experience with this.

Helium314 commented 3 months ago

Merging master should work normally. The master branch in this repo is currently up to date with SC.

On the offline tiles:

Should we guarantee they are stored for ApplicationData.DELETE_OLD_DATA_AFTER (14 days), so there are always tiles when there is downloaded map data? This requires keeping track of downloaded regions, but that should be simple. Using this options means that size of map tile database may exceed the value in MAP_TILECACHE_IN_MB pref. Or should we just download the tiles and allow deletion if the cache size is exceeded? I guess the longest unused tiles will go first, as there is an access date for each tile in the offline db (plus it makes sense).
I removed the tile download info logging, it will now only report the number and size of the tiles in the downloaded region, but not which / how many of them were already cached before download. No idea how to get this data from MapLibre, and I think it's not that important.
There is some setOfflineMapboxTileCountLimit option, which should invoke a callback when too many tiles are downloaded (and possibly will cancel the download, documentation is not clear). Anyway, I tried setting it to a very low number and still the callback wasn't triggered. Maybe we could ignore the callback, or set the tile count limit to a really high number to avoid unexpected issues later.

westnordost commented 3 months ago

Should we guarantee they are stored for ApplicationData.DELETE_OLD_DATA_AFTER (14 days), so there are always tiles when there is downloaded map data? This requires keeping track of downloaded regions, but that should be simple. Using this options means that size of map tile database may exceed the value in MAP_TILECACHE_IN_MB pref.

The technique is different, so the implementation can also be different. So, I would say, whatever is most straightforward to implement. If I had to choose, I would think it more important that the old data is only deleted after 14 days than that there is a MAP_TILECACHE_IN_MB setting at all(, or be respected closely).

westnordost commented 3 months ago

(The MAP_TILECACHE_IN_MB setting has always been somewhat... hmm... intransparent/useless to users because users have no idea how big in MB a given area is. I would not miss this setting. The setting has existed before I implemented offline map tiles download.)

Helium314 commented 3 months ago

Then I'll add this part to a todo section on top of the PR. I will need to store offline region id and dowloaded timestamp in a new table, and I think this is best to do just before merging the PR (and after merging master into this branch).

westnordost commented 2 months ago

I tested how much time it takes to transform the Pins and StyledElements to Geojson Features. The time it takes is negligible (0-4ms in my tests), so no need for any optimization that would cache the Features or anything like that.

I also moved the creation of the preset icon bitmaps (~400ms) and pin icon bitmaps (~800ms) into the Preloader, so it is only done once and immediately on startup in the background, which should speed up the time until the map is fully visible by one second or so. Also, I fixed a bug where the location dot would only appear after the next GPS update occured (5s), which made it look like it takes that long until the map is fully loaded. So, especially on startup, the map should feel a bit faster.

westnordost commented 2 months ago

I will write an own todo list here, as the starting comment has a lot of things that are not actually todos, too, and I am not sure with whether you are okay with me removing them. So, let's do a complete todo list in this comment.

Feel free to update this comment to either add todos or remove todos that have been solved

to do

merge current master
offline tiles download: add new database table and uncomment functionality
offline tiles download: add tests for DAO
offline tiles download: other related in-code todos
replace homebrew solution for quests and overlays with CustomGeometrySource (draft started by @Helium314)
replace quest circles with clusterable quest pins
camera: fix disabling navigation mode doesn't undo the tilt
camera: fix when tilt is used and map is scrolled up far enough, pointer pin points 180° wrong
camera: fix moving to user position after auto-zoom from opening a quest keeps camera padding, thus not centering the user's location
map style: fix oneway arrows are missing

to do SCEE

re-implement aerial imagery style: style json, adding and changing aerial imagery layer
(optional:) if quest pin size has considerable impact on performance with MapLibre, a SCEE setting to half the size of quest pins would be conceivable

upstream blockers

cosmetic rendering imperfection: maplibre-native#2217
cosmetic rendering ugliness: maplibre-native#2175 - if not fixed, can be worked around by not defining a halo
crash on 32bit devices: maplibre-native#2206 - should affect only old devices

Helium314 commented 2 months ago

I am not sure with whether you are okay with me removing them

Would be ok for things that are outdated or we don't want to touch anyway. I'll copy it to the top post.

adjust the fling behavior

Thanks for the reminder, now I finally can try it!

westnordost commented 2 months ago

I'll look at the oneway arrows in the map style next, plus look into the issues with the camera. I'll leave the offline tiles and CustomGeometrySource stuff to you. Regarding clustering, I guess that should come after the refactor of the QuestPinManager.

And regarding merge from master, ... uh ... maybe I do that after, unless you think it makes more sense if you do it.

Helium314 commented 2 months ago

And regarding merge from master, ... uh ... maybe I do that after, unless you think it makes more sense if you do it.

Probably better if you do it, as you have better overview about what is changed and where MapLibre things should go if they need to be moved

Helium314 commented 2 months ago

Offline tiles should be fine, only need to test whether

old offline regions are really deleted
old tiles are really removed if cache size is too large

westnordost commented 2 months ago

I think I will remove the local streetcomplete.json file. As long as for offline download, the streetcomplete.json needs to be pulled anyway from the URL, it seems to me it is only overhead to also have it locally.

Any resources used, like sprites (=oneway arrows) or glyphs (fonts) are also pulled from remote.

westnordost commented 2 months ago

Would you agree?

Helium314 commented 2 months ago

The only drawback I can think of is that theme switch may have issues when the new style is not yet downloaded. Otherwise I think it's fine.

westnordost commented 2 months ago

I'll next review the code so (except the areas I know are still worked on) and fix/change anything I notice along the way or add to the todos.

Helium314 commented 2 months ago

Maybe something to look into: on first download after install I didn't get any quests to show up, they came only after restarting the app. Further downloads were working, and I could not reproduce it by clearing app storage.

Helium314 commented 2 months ago

Putting the MapTilesDownloader in the cleaner was a bad idea... init calls MapLibre.getInstance(context), which only works from UI thread, so it crashes when done from cleaner worker (possibly works when SC is still in recents). Since MapLibre.getInstance(context) needs to be called before accessing the OfflineManager (for cleaning old regions / tiles), we need some alternative.

And another (minor) issue: After starting I don't see the undo button, even when there are not uploaded quests. I need to solve a new quest for it to appear.

westnordost commented 2 months ago

Hm, but the MapTilesDownloader is also the class that takes care of managing the data downloaded for offline. I'll look into it.

How do you detect a crash in the CleanerWorker when the app itself does not run?

Helium314 commented 2 months ago

I noticed old crash messages in Android Studio logcat view

westnordost commented 2 months ago

I made it so that exceptions are just caught and logged. If it is not executed on the main thread, and that is a prerequisite, there is not much we can do. (I also, unrelated to the fix, because all those callbacks were difficult to read, added extension functions that are suspending version of the original API so that the code can be written simply from top to bottom)

westnordost commented 2 months ago

And another (minor) issue: After starting I don't see the undo button, even when there are not uploaded quests. I need to solve a new quest for it to appear.

Already fixed on streetcomplete master

Helium314 commented 2 months ago

Maybe something to look into: on first download after install I didn't get any quests to show up, they came only after restarting the app. Further downloads were working, and I could not reproduce it by clearing app storage.

Its seems to be related to the style download, sometimes happened when starting while the phone is offline. But I guess it's fixed already by having the styles offline again.

westnordost commented 2 months ago

Oh, cool

Helium314 commented 2 months ago

When testing for the SDF icons I noticed that overlay icons seems to be blocked by quest pins, is this an intentional change?

westnordost commented 2 months ago

No. They just should block each other below a certain zoom level (they should not overlap that early). However, I think it is not possible to specify that icons in the same layer should block each other but disregard any icons in any other layers?

Helium314 commented 2 months ago

@westnordost what do you think about a new PR in the correct place, i.e. to SC? When I created this PR I wasn't sure whether this branch would be scrapped or evolve into the "real thing", and further I wanted to avoid attention (other people discussing / commenting here).

Now the PR is rather close to being done, with only a short todo list.

westnordost commented 2 months ago

🤷 I don't see why not (but I also don't see why)

Helium314 commented 2 months ago

fix move node functionality is somewhat broken (wrong positions) - something to do with padding

From my testing:

movement seems to start and calculate distance from center of the screen, but display the arrow from the correct position (yes, definitely something with padding)
moving node to the initial position and pressing ok correctly results in "please move it further", but when moving the wrong distance is shown again
works in overlays, but there is a way of getting the broken state as in quests (can't yet reproduce)
there is some way of getting into a different broken state, where the quest pin is large, but no quest form is shown and other pins are shown too. Not sure how, got it twice after turning the screen off and on again (or switching between apps); maybe this is related.

westnordost commented 2 months ago

I am fixing it right now. Sorry, I should have posted here that I am working on it.

Helium314 commented 2 months ago

offline tiles download: test whether offline region behavior works as intended (deleting and not exceeding size)

Part 1 tested: The initial offline db is about 15 MB (mostly glyphs for the style). After scrolling a lot, size was 27 MB. Then setting cache size to 10 MB shrank the db to 25 MB, so that's working.

Part 2 missing: clearing offline regions. I reduced the time until the regions get removed, lets see how the db has changed tomorow.

westnordost commented 2 months ago

I found some more issues which I believe must also exist for current master, related to changing between portrait and landscape mode while a form is open. However, I don't want to care about this too much because pending is a big refactor to use viewmodels and then jetpack compose anyway.

Helium314 commented 2 months ago

I just got a "StreetComplete keeps stopping" message by the OS, after having removed SC from recents around 18:50, and not using the phone after that until 20:34. Looks like it's related to the cleaner.

caught exception in MapTilesDownload at 19:28:38.549

``` Using MapView requires calling MapLibre.getInstance(Context context, String apiKey, WellKnownTileServer wellKnownTileServer) before inflating or creating the view. org.maplibre.android.exceptions.MapLibreConfigurationException: Using MapView requires calling MapLibre.getInstance(Context context, String apiKey, WellKnownTileServer wellKnownTileServer) before inflating or creating the view. at org.maplibre.android.MapLibre.validateMapbox(MapLibre.java:247) at org.maplibre.android.MapLibre.getTileServerOptions(MapLibre.java:150) at org.maplibre.android.storage.FileSource.(FileSource.java:375) at org.maplibre.android.storage.FileSource.getInstance(FileSource.java:95) at org.maplibre.android.offline.OfflineManager.(OfflineManager.kt:107) at org.maplibre.android.offline.OfflineManager.(OfflineManager.kt:0) at org.maplibre.android.offline.OfflineManager$Companion.getInstance(OfflineManager.kt:586) at de.westnordost.streetcomplete.data.maptiles.MapTilesDownloader.deleteRegionsOlderThan(MapTilesDownloader.kt:34) at de.westnordost.streetcomplete.data.Cleaner$cleanOld$1.invokeSuspend(Cleaner.kt:37) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ```

later crashes in background

``` 2024-04-17 20:29:48.643 28576-28576 AndroidRuntime pid-28576 E FATAL EXCEPTION: main Process: de.westnordost.streetcomplete, PID: 28576 java.lang.IllegalMonitorStateException at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1291) at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:466) at org.maplibre.android.storage.FileSource.unlockPathLoaders(Unknown Source:2) at org.maplibre.android.storage.FileSource.access$100(Unknown Source:0) at org.maplibre.android.storage.FileSource$FileDirsPathsTask.onPostExecute(SourceFile:0) at org.maplibre.android.storage.FileSource$FileDirsPathsTask.onPostExecute(SourceFile:0) at android.os.AsyncTask.finish(AsyncTask.java:755) at android.os.AsyncTask.access$900(AsyncTask.java:192) at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:772) at android.os.Handler.dispatchMessage(Handler.java:107) at android.os.Looper.loop(Looper.java:214) at android.app.ActivityThread.main(ActivityThread.java:7356) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:491) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:940) 2024-04-17 20:30:19.187 30284-30284 AndroidRuntime pid-30284 E FATAL EXCEPTION: main Process: de.westnordost.streetcomplete, PID: 30284 java.lang.IllegalMonitorStateException at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1291) at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:466) at org.maplibre.android.storage.FileSource.unlockPathLoaders(Unknown Source:2) at org.maplibre.android.storage.FileSource.access$100(Unknown Source:0) at org.maplibre.android.storage.FileSource$FileDirsPathsTask.onPostExecute(SourceFile:0) at org.maplibre.android.storage.FileSource$FileDirsPathsTask.onPostExecute(SourceFile:0) at android.os.AsyncTask.finish(AsyncTask.java:755) at android.os.AsyncTask.access$900(AsyncTask.java:192) at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:772) at android.os.Handler.dispatchMessage(Handler.java:107) at android.os.Looper.loop(Looper.java:214) at android.app.ActivityThread.main(ActivityThread.java:7356) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:491) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:940) ```

after plugging phone, that's when I got the "StreetComplete keeps stopping" message

``` 2024-04-17 20:34:02.831 30393-30418 WM-WorkerFactory de.westnordost.streetcomplete E Could not instantiate de.westnordost.streetcomplete.data.CleanerWorker java.lang.NoSuchMethodException: de.westnordost.streetcomplete.data.CleanerWorker. [class android.content.Context, class androidx.work.WorkerParameters] at java.lang.Class.getConstructor0(Class.java:2332) at java.lang.Class.getDeclaredConstructor(Class.java:2170) at androidx.work.WorkerFactory.createWorkerWithDefaultFallback(Unknown Source:60) at androidx.work.impl.WorkerWrapper.runWorker(Unknown Source:309) at androidx.work.impl.WorkerWrapper.run(Unknown Source:8) at androidx.work.impl.utils.SerialExecutorImpl$Task.run(Unknown Source:2) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.lang.Thread.run(Thread.java:919) ```

In case that's not enough, when starting SC I got this native crash

``` 2024-04-17 20:39:58.989 28440-28440 libc de.westnordost.streetcomplete.debug A Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xf8 in tid 28440 (tcomplete.debug), pid 28440 (tcomplete.debug) 2024-04-17 20:39:59.117 31292-31292 DEBUG crash_dump64 A pid: 28440, tid: 28440, name: tcomplete.debug >>> de.westnordost.streetcomplete.debug <<< ---------------------------- PROCESS STARTED (31294) for package de.westnordost.streetcomplete.debug ---------------------------- 2024-04-17 20:39:59.539 31292-31292 DEBUG crash_dump64 A #00 pc 000000000083d238 /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/lib/arm64/libmaplibre.so (BuildId: 17dce6c1df7723a0f1c2f7eea3f02b2b0413e615) 2024-04-17 20:39:59.539 31292-31292 DEBUG crash_dump64 A #01 pc 0000000000839e70 /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/lib/arm64/libmaplibre.so (BuildId: 17dce6c1df7723a0f1c2f7eea3f02b2b0413e615) 2024-04-17 20:39:59.539 31292-31292 DEBUG crash_dump64 A #02 pc 000000000044abf8 /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/lib/arm64/libmaplibre.so (mbgl::android::AndroidRendererFrontend::AndroidRendererFrontend(mbgl::android::MapRenderer&)+96) (BuildId: 17dce6c1df7723a0f1c2f7eea3f02b2b0413e615) 2024-04-17 20:39:59.539 31292-31292 DEBUG crash_dump64 A #03 pc 000000000047ddf0 /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/lib/arm64/libmaplibre.so (mbgl::android::NativeMapView::NativeMapView(_JNIEnv&, jni::Object const&, jni::Object const&, jni::Object const&, float, unsigned char)+212) (BuildId: 17dce6c1df7723a0f1c2f7eea3f02b2b0413e615) 2024-04-17 20:39:59.539 31292-31292 DEBUG crash_dump64 A #04 pc 000000000049912c /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/lib/arm64/libmaplibre.so (_ZN3jni8MakePeerIN4mbgl7android13NativeMapViewEJRKNS_6ObjectIS3_EERKNS4_INS2_10FileSourceEEERKNS4_INS2_11MapRendererEEEfhEEENSt6__ndk110unique_ptrIT_NSG_14default_deleteISI_EEEER7_JNIEnvDpT0_+92) (BuildId: 17dce6c1df7723a0f1c2f7eea3f02b2b0413e615) 2024-04-17 20:39:59.539 31292-31292 DEBUG crash_dump64 A #05 pc 000000000048f8d8 /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/lib/arm64/libmaplibre.so (_ZZNK3jni16NativePeerHelperIN4mbgl7android13NativeMapViewES3_FNSt6__ndk110unique_ptrIS3_NS4_14default_deleteIS3_EEEER7_JNIEnvRKNS_6ObjectIS3_EERKNSB_INS2_10FileSourceEEERKNSB_INS2_11MapRendererEEEfhEE15MakeInitializerERKNS_5FieldIS3_lEEPKcPSN_ENKUlSA_RSC_SW_RSG_RSK_RfRhE_clESA_SW_SW_SX_SY_SZ_S10_+160) (BuildId: 17dce6c1df7723a0f1c2f7eea3f02b2b0413e615) 2024-04-17 20:39:59.540 31292-31292 DEBUG crash_dump64 A #06 pc 000000000048f810 /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/lib/arm64/libmaplibre.so (_ZZN3jni17NativeMethodMakerIMZNKS_16NativePeerHelperIN4mbgl7android13NativeMapViewES4_FNSt6__ndk110unique_ptrIS4_NS5_14default_deleteIS4_EEEER7_JNIEnvRKNS_6ObjectIS4_EERKNSC_INS3_10FileSourceEEERKNSC_INS3_11MapRendererEEEfhEE15MakeInitializerERKNS_5FieldIS4_lEEPKcPSO_EUlSB_RSD_SX_RSH_RSL_RfRhE_KFvSB_SX_SX_SY_SZ_S10_S11_EJEEclIS12_EEDaSV_RKT_ENUlPSA_PNS_7jobjectES1C_S1C_S1C_fhE_8__invokeES1A_S1C_S1C_S1C_S1C_fh+92) (BuildId: 17dce6c1df7723a0f1c2f7eea3f02b2b0413e615) 2024-04-17 20:39:59.540 31292-31292 DEBUG crash_dump64 A #07 pc 000000000048fa28 /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/lib/arm64/libmaplibre.so (_ZZN3jni16MakeNativeMethodIZNS_17NativeMethodMakerIMZNKS_16NativePeerHelperIN4mbgl7android13NativeMapViewES5_FNSt6__ndk110unique_ptrIS5_NS6_14default_deleteIS5_EEEER7_JNIEnvRKNS_6ObjectIS5_EERKNSD_INS4_10FileSourceEEERKNSD_INS4_11MapRendererEEEfhEE15MakeInitializerERKNS_5FieldIS5_lEEPKcPSP_EUlSC_RSE_SY_RSI_RSM_RfRhE_KFvSC_SY_SY_SZ_S10_S11_S12_EJEEclIS13_EEDaSW_RKT_EUlPSB_PNS_7jobjectES1D_S1D_S1D_fhE_EEDaSW_SW_S1A_PNS6_9enable_ifIXsr3std8is_classIS18_EE5valueEvE4typeEENUlS1B_DpT_E_8__invokeIJS1D_S1D_S1D_S1D_fhEEEDaS1B_S1K_+44) (BuildId: 17dce6c1df7723a0f1c2f7eea3f02b2b0413e615) 2024-04-17 20:39:59.540 31292-31292 DEBUG crash_dump64 A #15 pc 00000000000a70ea [anon:dalvik-classes3.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk!classes3.dex] (org.maplibre.android.maps.NativeMapView.+70) 2024-04-17 20:39:59.540 31292-31292 DEBUG crash_dump64 A #18 pc 00000000000a63e8 [anon:dalvik-classes3.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk!classes3.dex] (org.maplibre.android.maps.MapView.initialiseDrawingSurface+208) 2024-04-17 20:39:59.540 31292-31292 DEBUG crash_dump64 A #21 pc 00000000000a664c [anon:dalvik-classes3.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk!classes3.dex] (org.maplibre.android.maps.MapView.initialize+80) 2024-04-17 20:39:59.541 31292-31292 DEBUG crash_dump64 A #24 pc 00000000000a61fc [anon:dalvik-classes3.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk!classes3.dex] (org.maplibre.android.maps.MapView.+116) 2024-04-17 20:39:59.543 31292-31292 DEBUG crash_dump64 A #64 pc 000000000023c86a [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.Fragment.onCreateView+10) 2024-04-17 20:39:59.543 31292-31292 DEBUG crash_dump64 A #67 pc 000000000023e718 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.Fragment.performCreateView+48) 2024-04-17 20:39:59.543 31292-31292 DEBUG crash_dump64 A #70 pc 0000000000239012 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentStateManager.createView+426) 2024-04-17 20:39:59.543 31292-31292 DEBUG crash_dump64 A #73 pc 0000000000239778 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentStateManager.moveToExpectedState+244) 2024-04-17 20:39:59.544 31292-31292 DEBUG crash_dump64 A #76 pc 000000000023ad10 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentStore.moveToExpectedState+60) 2024-04-17 20:39:59.544 31292-31292 DEBUG crash_dump64 A #79 pc 00000000002376ce [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentManager.moveToState+54) 2024-04-17 20:39:59.544 31292-31292 DEBUG crash_dump64 A #82 pc 000000000023690e [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentManager.dispatchStateChange+18) 2024-04-17 20:39:59.544 31292-31292 DEBUG crash_dump64 A #85 pc 00000000002369a6 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentManager.dispatchViewCreated+2) 2024-04-17 20:39:59.544 31292-31292 DEBUG crash_dump64 A #88 pc 000000000023ed46 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.Fragment.performViewCreated+38) 2024-04-17 20:39:59.544 31292-31292 DEBUG crash_dump64 A #91 pc 00000000002390e4 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentStateManager.createView+636) 2024-04-17 20:39:59.544 31292-31292 DEBUG crash_dump64 A #94 pc 0000000000239778 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentStateManager.moveToExpectedState+244) 2024-04-17 20:39:59.544 31292-31292 DEBUG crash_dump64 A #97 pc 000000000023ad10 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentStore.moveToExpectedState+60) 2024-04-17 20:39:59.545 31292-31292 DEBUG crash_dump64 A #100 pc 00000000002376ce [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentManager.moveToState+54) 2024-04-17 20:39:59.545 31292-31292 DEBUG crash_dump64 A #103 pc 000000000023690e [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentManager.dispatchStateChange+18) 2024-04-17 20:39:59.545 31292-31292 DEBUG crash_dump64 A #106 pc 000000000023639e [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentManager.dispatchActivityCreated+22) 2024-04-17 20:39:59.545 31292-31292 DEBUG crash_dump64 A #109 pc 0000000000232458 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentController.dispatchActivityCreated+8) 2024-04-17 20:39:59.545 31292-31292 DEBUG crash_dump64 A #112 pc 0000000000231548 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.fragment.app.FragmentActivity.onStart+40) 2024-04-17 20:39:59.545 31292-31292 DEBUG crash_dump64 A #115 pc 0000000000195bf8 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (androidx.appcompat.app.AppCompatActivity.onStart) 2024-04-17 20:39:59.545 31292-31292 DEBUG crash_dump64 A #118 pc 0000000000470458 [anon:dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.debug-2NYVvRrm4QAUZiqEeZchiw==/base.apk] (de.westnordost.streetcomplete.screens.MainActivity.onStart) ```

Starting SC again worked normally. I'll leave SC running in background, hoping that the cleaner will run.

westnordost commented 2 months ago

The last one looks like a MapLibre something-something. Crash-dumps will hopefully become more useful once https://github.com/maplibre/maplibre-native/issues/2282 is done.

The cleaner-stuff relates to that other thread where I mentioned that maybe we should give up on manually cleaning the cache because it only works if MapLibre has been initialized on the main thread. TBH I think maybe it is not that complex to implement in MapLibre proper. As in the OfflineManager, there is already an option to limit the cache size, I am sure that MapLibre already internally remembers which tile has been downloaded when because it would make most sense to first evict the oldest downloaded tiles from the cache.

The ReentrantLock thing will be very hard to find, as not in a single line in the stack trace, StreetComplete is mentioned.

This and the coming week, I will not work on this PR.

westnordost commented 2 months ago

Stack trace number three might also be related to Koin. It looks as if the system was unable to initialize the worker at all.

Helium314 commented 2 months ago

I reduced the time until the regions get removed, lets see how the db has changed tomorow.

Did not work. SC was not closed and there were no crashes, but to offline region cleanup happened.

It worked correctly when I added mapTilesDownloader.deleteRegionsOlderThan to startup code. Maybe we could just call it every time the map is initialized? It's not doing any heavy work, so should not noticeably slow down anything.

westnordost commented 2 months ago

pre-6 has been released - https://github.com/maplibre/maplibre-native/releases/tag/android-v11.0.0-pre6

added debug symbols for Android to help debugging native crashes
maybe https://github.com/maplibre/maplibre-native/issues/2217 is fixed

westnordost commented 2 months ago

It worked correctly when I added mapTilesDownloader.deleteRegionsOlderThan to startup code. Maybe we could just call it every time the map is initialized? It's not doing any heavy work, so should not noticeably slow down anything.

I guess that would be the workaround. I.e. not have the OfflineManager in the Cleaner at all, but do it on initializing the map. This will put some IO work at startup though, i.e. at the time at which we do a lot of IO anyway.

Helium314 commented 2 months ago

This will put some IO work at startup though, i.e. at the time at which we do a lot of IO anyway.

It's not really important to be done immediately on start, we could e.g. delay it by 30 seconds.

Helium314 / SCEE

Maplibre #516

to do

to do SCEE

upstream blockers

to do

to do SCEE

upstream blockers