maplibre / maplibre-native

MapLibre Native - Interactive vector tile maps for iOS, Android and other platforms.
https://maplibre.org
BSD 2-Clause "Simplified" License
923 stars 279 forks source link

Crash with android-v11.0.0-pre4 (Samsung Galaxy S4 Mini, LineageOS 16.0, Android 9) #2206

Open Helium314 opened 4 months ago

Helium314 commented 4 months ago

Describe the bug App crashes on start

To Reproduce Steps to reproduce the behavior:

  1. Have app with working maplibre 10.0.2
  2. Upgrade to 11.0.0-pre4
  3. Start app
  4. See crash (looks like it happens after style is loaded, but I don't see any map tiles are rendered)
19:32:42.633 libc                     A  Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x8 in tid 32277 (GLThread 3086), pid 32229 (mplete.ml.debug)
19:32:43.117 DEBUG                    A  pid: 32229, tid: 32277, name: GLThread 3086  >>> de.westnordost.streetcomplete.ml.debug <<<
19:32:43.205 DEBUG                    A      #21 pc 00622405  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #22 pc 0063909d  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #23 pc 00637eff  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #24 pc 004a79e9  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #25 pc 004a67c7  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #26 pc 004b7595  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.206 DEBUG                    A      #27 pc 004c9483  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.206 DEBUG                    A      #28 pc 004c8817  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.206 DEBUG                    A      #29 pc 0037333f  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so (mbgl::android::MapRenderer::render(_JNIEnv&)+194)
19:32:43.206 DEBUG                    A      #30 pc 003751a9  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so (_ZZN3jni16MakeNativeMethodIZNS_17NativeMethodMakerIMZNS_30NativePeerMemberFunctionMethodIMN4mbgl7android11MapRendererEFvR7_JNIEnvEXadL_ZNS5_6renderES7_EEEclIS5_S5_vEEDaRKNS_5FieldIT0_xEEEUlS7_RNS_6ObjectIS5_EEE_KFvS7_SJ_EJEEclISK_EEDaPKcRKT_EUlPS6_PNS_7jobjectEE_EEDaSQ_SQ_ST_PNSt6__ndk19enable_ifIXsr3std8is_classISR_EE5valueEvE4typeEENUlSU_DpT_E_8__invokeIJSW_EEEDaSU_S14_+28)
19:32:43.206 DEBUG                    A      #39 pc 0056e994  /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/base.apk (deleted) (org.maplibre.android.maps.renderer.MapRenderer.onDrawFrame+12)
19:32:43.206 DEBUG                    A      #45 pc 0056f8a0  /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/base.apk (deleted) (org.maplibre.android.maps.renderer.glsurfaceview.GLSurfaceViewMapRenderer.onDrawFrame)
19:32:43.207 DEBUG                    A      #51 pc 00570406  /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/base.apk (deleted) (org.maplibre.android.maps.renderer.glsurfaceview.MapLibreGLSurfaceView$GLThread.guardedRun+994)
19:32:43.207 DEBUG                    A      #57 pc 00570960  /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/base.apk (deleted) (org.maplibre.android.maps.renderer.glsurfaceview.MapLibreGLSurfaceView$GLThread.run+48)

Expected behavior Normal startup

Platform information (please complete the following information):

Additional context Works on other devices. S4 Mini supports OpenGL ES 3.0 in case the question comes up.

louwers commented 4 months ago

Thanks, we're looking into it.

louwers commented 4 months ago

@Helium314 Could you tell what the model number of the phone is?

Should be in the settings somewhere.

To debug this we need to buy this phone (luckily they are not expensive anymore).

Helium314 commented 4 months ago

The model is GT-9195. Anything I can help you with before you really need to buy one of these?

Helium314 commented 4 months ago

Maybe there is some other issue... I just tested it on my other S4 Mini VE (9195I), which has Snapdragon 410 instead of 400, and got the same crash. Both use the older 32 bit armeabi-v7a (even though SD410 could run arm64-v8a).

mwilsnd commented 4 months ago

Interesting, that might be a 32-bit problem then. If you can, could you build the Android demo application in this repository and run it on these devices? Building everything in debug mode will also hopefully provide a full call stack with C++ symbols.

stefankarschti commented 3 months ago

Describe the bug App crashes on start

To Reproduce Steps to reproduce the behavior:

  1. Have app with working maplibre 10.0.2
  2. Upgrade to 11.0.0-pre4
  3. Start app
  4. See crash (looks like it happens after style is loaded, but I don't see any map tiles are rendered)
19:32:42.633 libc                     A  Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x8 in tid 32277 (GLThread 3086), pid 32229 (mplete.ml.debug)
19:32:43.117 DEBUG                    A  pid: 32229, tid: 32277, name: GLThread 3086  >>> de.westnordost.streetcomplete.ml.debug <<<
19:32:43.205 DEBUG                    A      #21 pc 00622405  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #22 pc 0063909d  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #23 pc 00637eff  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #24 pc 004a79e9  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #25 pc 004a67c7  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.205 DEBUG                    A      #26 pc 004b7595  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.206 DEBUG                    A      #27 pc 004c9483  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.206 DEBUG                    A      #28 pc 004c8817  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so
19:32:43.206 DEBUG                    A      #29 pc 0037333f  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so (mbgl::android::MapRenderer::render(_JNIEnv&)+194)
19:32:43.206 DEBUG                    A      #30 pc 003751a9  /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/lib/arm/libmapbox-gl.so (_ZZN3jni16MakeNativeMethodIZNS_17NativeMethodMakerIMZNS_30NativePeerMemberFunctionMethodIMN4mbgl7android11MapRendererEFvR7_JNIEnvEXadL_ZNS5_6renderES7_EEEclIS5_S5_vEEDaRKNS_5FieldIT0_xEEEUlS7_RNS_6ObjectIS5_EEE_KFvS7_SJ_EJEEclISK_EEDaPKcRKT_EUlPS6_PNS_7jobjectEE_EEDaSQ_SQ_ST_PNSt6__ndk19enable_ifIXsr3std8is_classISR_EE5valueEvE4typeEENUlSU_DpT_E_8__invokeIJSW_EEEDaSU_S14_+28)
19:32:43.206 DEBUG                    A      #39 pc 0056e994  /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/base.apk (deleted) (org.maplibre.android.maps.renderer.MapRenderer.onDrawFrame+12)
19:32:43.206 DEBUG                    A      #45 pc 0056f8a0  /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/base.apk (deleted) (org.maplibre.android.maps.renderer.glsurfaceview.GLSurfaceViewMapRenderer.onDrawFrame)
19:32:43.207 DEBUG                    A      #51 pc 00570406  /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/base.apk (deleted) (org.maplibre.android.maps.renderer.glsurfaceview.MapLibreGLSurfaceView$GLThread.guardedRun+994)
19:32:43.207 DEBUG                    A      #57 pc 00570960  /dev/ashmem/dalvik-classes.dex extracted in memory from /data/app/de.westnordost.streetcomplete.ml.debug-X6eOgc-FLo8RJAY1-LJMlw==/base.apk (deleted) (org.maplibre.android.maps.renderer.glsurfaceview.MapLibreGLSurfaceView$GLThread.run+48)

Expected behavior Normal startup

Platform information (please complete the following information):

  • OS: LineageOS 16.0 / Android 9
  • Platform: Android
  • Version: 9
  • Device: Samsung Galaxy S4 Mini

Additional context Works on other devices. S4 Mini supports OpenGL ES 3.0 in case the question comes up.

Is the tombstone file generated on the phone? Its path is written in logs if so.

Helium314 commented 3 months ago

This one? tombstone_17.txt

stefankarschti commented 3 months ago

yes, this is the file. I was hoping to see symbols, but it's not helpful. we need to reproduce the crash with a debug library.

Helium314 commented 3 months ago

How do you build the debug version? I get CMake 3.19 or higher is required. You are running version 3.18.1-g262b901, but dependencies.gradle actually says cmakeVersion : '3.18.1+', and when I change it to 3.19.1+ there is some weird The source directory error message.

mwilsnd commented 3 months ago

Can you share a snippet of the error being generated? The gradle file needs to get updated but manually installing a newer version of CMake should work.

You might need to clear out the old files generated by CMake. You can try deleting the following, if they exist: platform/android/build, platform/android/MapboxGLAndroidSDK/build and platform/android/MapboxGLAndroidSDKTestApp/build

Helium314 commented 3 months ago

Thanks! So, onwards to the next error I guess: License not found for target: mbgl-vendor-boost (from license.cmake:24)

Is there a debug apk I can download? I fear building this might take a lot of time...

mwilsnd commented 3 months ago

Sounds like you might be missing the submodules. Run git submodule update --init --recursive to make sure you have them all downloaded.

louwers commented 3 months ago

@Helium314 If you are still having trouble I can make a debug build and share it with you.

Helium314 commented 3 months ago

The crash happens when I choose "add markers from json api". Everything above seems to work. log.txt tombstone_25.txt tombstone_26.txt

Edit: zooming in the simple map also crashes

westnordost commented 3 months ago

In case this crash is not specific to the device but generally occurs for devices that only support the armeabi-v7a, for what it's worth, I just noticed that there are some relatively new devices that only support that ABI, for example apparently and most prominently the Samsung Galaxy A13 (and variations).

louwers commented 3 months ago

@westnordost I'll try another 32-bit device.

louwers commented 3 months ago

@westnordost From what I can tell the Samsung Galaxy A13 uses a MediaTek MT6833 CPU which uses a 64-bit set.

louwers commented 3 months ago

AWS Device Farm has a Samsung Galaxy J7 that I could test on (although it runs Android 8, so it would be some work).

Helium314 commented 3 months ago

As I mentioned, it does not only happen on S4 Mini, but also on S4 Mini VE, which is 2 years newer and has a 64 bit processor (SD410) but can for other reasons only run a 32 bit OS.

westnordost commented 3 months ago

@louwers That information originated from the Google Play console. That is the frontend for application publishers on Google Play, it offers all kind of statistics. In the Device catalogue, I am able to see which devices can run my app, I can see the number of installs of my app on devices for each model and finally I can filter by various parameters, one of these is the ABI.

When I filter by the minimum system requirements of MapLibre: ABI = armeabi-v7a, Android SDK ≥ 21, OpenGL ES ≥ 3.0, I get the following list. I reordered it by Android SDK versions descending. Maybe it is helpful.

devices.csv

Helium314 commented 3 months ago

The crash happens when I choose "add markers from json api". Everything above seems to work. log.txt tombstone_25.txt tombstone_26.txt

Edit: zooming in the simple map also crashes

Is there any more useful info I can provide? Specific inputs, specific parts of the test app, ...

westnordost commented 2 months ago

For pre-06, debug symbols have been added. I suspect that debugging the source of the issue will be easier now.

Helium314 commented 2 months ago

I hope this contains more usable information: crash.zip

mwilsnd commented 2 months ago

Looks like it crashed trying to link a permuted symbol layer shader. Both devices run Adreno 3xx GPUs which apparently have some pretty cursed conformance issues with OpenGL ES 3.0. I'll try and get one of these devices to debug the shaders with, but it looks like there are a few known issues with those drivers.

mwilsnd commented 2 months ago

Update: I got an S4 mini (SGH-I257M) which has an Adreno 3xx GPU and appears to be identical to the SKUs you listed. I put Lineage on it and was able to replicate the crash. Stopping the symbol layer from rendering resulted in an otherwise correct looking map. Screenshot_20240502-144713

I was also able to confirm it's the shader compilation (glLinkProgram) crashing, so I can try and debug the shaders now to see what exactly is going wrong.

I do have to wonder how many devices are actually out there with an Adreno 3xx GPU and an Android API level high enough to work with MapLibre. The stock ROM on this device is API level 19 which isn't high enough so you have to put a custom image like Lineage on it to even end up in this situation. For now, I think we should try and run on Adreno 3xx if it isn't too difficult and/or there are other devices out there that can run MapLibre without custom OS installs.

westnordost commented 2 months ago

I do have to wonder how many devices are actually out there with an Adreno 3xx GPU and an Android API level high enough to work with MapLibre.

I have access to this data. Here is a list of all devices that use a Adreno 3xx GPU (, are supported by my app, i.e. have minimum Android API level of 21) and support at least OpenGL ES 3.0:

devices.csv

westnordost commented 2 months ago

Note that this list appears as if it contains a lot of duplicates, e.g. there are 18 devices named Samsung Galaxy S4 in there. This is because each configuration is listed separately - different batches of Galaxy S4s have apparently different amounts of RAM, different CPU, GPU and chipset.

mwilsnd commented 2 months ago

Thank you for this list, even with duplicates there are still quite a few devices with 304/305 GPUs out there. I'll provide further updates in a future PR linking to this issue when I track down the root cause in the shaders.

kodebach commented 1 week ago

We are seeing this issue in our app as well. We are also seeing it on devices with other GPUs, however, they are all still Samsung devices. We've also seen a segfault in Google Play Pre-Launch Reports on a Motorola G20, but we were not able to confirm that it was caused by Maplibre.

This is the current crash statistic from our Sentry instance

device model crashes in last 90 days
SM-A600FN 108
SM-A336B 39
SM-A405FN 36
SM-A325F 56
SM-A320FL 212
SM-G525F 584

To frame the number of crashes seen per device: This is a private app deployed to fewer than 500 devices, all of them Samsung phones. The list above is the complete list of devices in use, so as far as we can tell, it is not hardware specific (unless it is a general Samsung issue).

With Maplibre 11 we finally got a more detailed stack trace

OS Version: Android 13 (TP1A.220624.014.G525FXXSACXCA)
Report Version: 104

Exception Type: Unknown (SIGSEGV)

Application Specific Information:
Segfault

Thread 0 Crashed:
0   split_config.arm64_v8a.apk      0x70a5f1d3ec        std::__ndk1::__split_buffer<T>::~__split_buffer
1   split_config.arm64_v8a.apk      0x70a5dfcacc        mbgl::android::MapRenderer::render
2   split_config.arm64_v8a.apk      0x70a5dff5fc        jni::MakeNativeMethod<T>::lambda::__invoke<T>
3   base.odex                       0x70bea5c2d4        <unknown> + 484234871508

Not sure if that is helpful in any way, __split_buffer seems to be a an internal class of the C++ stdlib used in std::vector and possibly others. But I did not see any std::vector instances in MapRenderer.

Sentry also shows the __split_buffer frame as (although I'm not sure how it got that information)

std::__ndk1::__split_buffer<mbgl::util::DefaultStyle, std::__ndk1::allocator<mbgl::util::DefaultStyle>&>::~__split_buffer()

That would point to

https://github.com/maplibre/maplibre-native/blob/5d5bd84d95f3692fa1bf05931cbf2fd1e222a6ce/src/mbgl/util/tile_server_options.cpp#L34

But I did not check how this is connected to MapRenderer (if at all).

Helium314 commented 1 week ago

@kodebach I'm not sure it's the same crash. The one I see is always there on GT-9195 and GT-9195I and makes the app unusable for those phones with MapLibre 11, but I did never observed it on SM-A320FL.

kodebach commented 1 week ago

You're right, I re-read the comments in this thread again and it is probably a different issue. I've created #2579.