Crash - org.chromium.base.PowerMonitor.a - java.util.ConcurrentModificationException

[gsarris]

Hello!

We are facing the following crash in our production app.

Fatal Exception: org.chromium.base.JniAndroid$UncaughtExceptionException: Native stack trace:
#00 pc 0x0000000003e8226f /data/app/~~-ja5jhRo4kb4D5H61zVY9A==/com.google.android.trichromelibrary_631211833-F6tvFEzV9ClCYMKg8o5AEg==/base.apk (offset 0x8dc000)
#01 pc 0x0000000002cf7f53 /data/app/~~-ja5jhRo4kb4D5H61zVY9A==/com.google.android.trichromelibrary_631211833-F6tvFEzV9ClCYMKg8o5AEg==/base.apk (offset 0x8dc000)
#02 pc 0x0000000002cf8df7 /data/app/~~-ja5jhRo4kb4D5H61zVY9A==/com.google.android.trichromelibrary_631211833-F6tvFEzV9ClCYMKg8o5AEg==/base.apk (offset 0x8dc000)
#03 pc 0x0000000003d12a37 /data/app/~~-ja5jhRo4kb4D5H61zVY9A==/com.google.android.trichromelibrary_631211833-F6tvFEzV9ClCYMKg8o5AEg==/base.apk (offset 0x8dc000)
#04 pc 0x0000000002c635a7 /data/app/~~-ja5jhRo4kb4D5H61zVY9A==/com.google.android.trichromelibrary_631211833-F6tvFEzV9ClCYMKg8o5AEg==/base.apk (offset 0x8dc000)
#05 pc 0x0000000002ce973b /data/app/~~-ja5jhRo4kb4D5H61zVY9A==/com.google.android.trichromelibrary_631211833-F6tvFEzV9ClCYMKg8o5AEg==/base.apk (offset 0x8dc000)
#06 pc 0x0000000002cbaafb /data/app/~~-ja5jhRo4kb4D5H61zVY9A==/com.google.android.trichromelibrary_631211833-F6tvFEzV9ClCYMKg8o5AEg==/base.apk (offset 0x8dc000)
#07 pc 0x0000000002cba63b /data/app/~~-ja5jhRo4kb4D5H61zVY9A==/com.google.android.trichromelibrary_631211833-F6tvFEzV9ClCYMKg8o5AEg==/base.apk (offset 0x8dc000)

       at org.chromium.base.JniAndroid.handleException(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:21)
       at J.N.M1Y_XVCN(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833)
       at org.chromium.content.browser.BrowserStartupControllerImpl.d(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:110)
       at WV.h5.run(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:180)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:487)
       at java.util.concurrent.FutureTask.run(FutureTask.java:264)
       at org.chromium.base.task.PostTask.d(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:11)
       at org.chromium.android_webview.AwBrowserProcess.j(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:32)
       at com.android.webview.chromium.E.d(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:188)
       at com.android.webview.chromium.E.b(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:42)
       at com.android.webview.chromium.E.f(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:4)
       at com.android.webview.chromium.WebViewChromiumFactoryProvider.l(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:8)
       at com.android.webview.chromium.WebViewChromium.init(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:97)
       at android.webkit.WebView.<init>(WebView.java:443)
       at android.webkit.WebView.<init>(WebView.java:363)
       at android.webkit.WebView.<init>(WebView.java:345)
       at android.webkit.WebView.<init>(WebView.java:332)
       at android.webkit.WebView.<init>(WebView.java:322)
       at com.tealium.tagmanagementdispatcher.d.invoke(SourceFile:1)
       at com.tealium.tagmanagementdispatcher.e$b.invokeSuspend(SourceFile:47)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
       at android.os.Handler.handleCallback(Handler.java:942)
       at android.os.Handler.dispatchMessage(Handler.java:99)
       at android.os.Looper.loopOnce(Looper.java:211)
       at android.os.Looper.loop(Looper.java:300)
       at android.app.ActivityThread.main(ActivityThread.java:8410)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:559)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:954)

Caused by java.util.ConcurrentModificationException:
       at android.util.ArrayMap.put(ArrayMap.java:623)
       at android.os.PowerManager.addThermalStatusListener(PowerManager.java:2471)
       at android.os.PowerManager.addThermalStatusListener(PowerManager.java:2443)
       at org.chromium.base.PowerMonitor.a(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:88)
       at org.chromium.base.PowerMonitor.isBatteryPower(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:5)
       at J.N.M1Y_XVCN(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833)
       at org.chromium.content.browser.BrowserStartupControllerImpl.d(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:110)
       at WV.h5.run(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:180)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:487)
       at java.util.concurrent.FutureTask.run(FutureTask.java:264)
       at org.chromium.base.task.PostTask.d(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:11)
       at org.chromium.android_webview.AwBrowserProcess.j(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:32)
       at com.android.webview.chromium.E.d(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:188)
       at com.android.webview.chromium.E.b(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:42)
       at com.android.webview.chromium.E.f(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:4)
       at com.android.webview.chromium.WebViewChromiumFactoryProvider.l(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:8)
       at com.android.webview.chromium.WebViewChromium.init(chromium-TrichromeWebViewGoogle6432.aab-stable-631211833:97)
       at android.webkit.WebView.<init>(WebView.java:443)
       at android.webkit.WebView.<init>(WebView.java:363)
       at android.webkit.WebView.<init>(WebView.java:345)
       at android.webkit.WebView.<init>(WebView.java:332)
       at android.webkit.WebView.<init>(WebView.java:322)
       at com.tealium.tagmanagementdispatcher.d.invoke(SourceFile:1)
       at com.tealium.tagmanagementdispatcher.e$b.invokeSuspend(SourceFile:47)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
       at android.os.Handler.handleCallback(Handler.java:942)
       at android.os.Handler.dispatchMessage(Handler.java:99)
       at android.os.Looper.loopOnce(Looper.java:211)
       at android.os.Looper.loop(Looper.java:300)
       at android.app.ActivityThread.main(ActivityThread.java:8410)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:559)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:954)

Crashes are happening in the background and we cannot reproduce.

We use the following libs:

api("com.tealium:kotlin-core:${rootProject.extra["tealiumCoreVersion"]}") api("com.tealium:kotlin-remotecommand-dispatcher:${rootProject.extra["tealiumDispatchersVersion"]}") api("com.tealium:kotlin-autotracking:${rootProject.extra["tealiumAutoTrackingVersion"]}") api("com.tealium:kotlin-lifecycle:${rootProject.extra["tealiumLifecycleVersion"]}") api("com.tealium:kotlin-tagmanagement-dispatcher:${rootProject.extra["tealiumTagManagementVersion"]}") api("com.tealium:kotlin-collect-dispatcher:${rootProject.extra["tealiumCollectDispatcherVersion"]}") api("com.tealium:kotlin-ad-identifier:${rootProject.extra["tealiumAdIdentifierVersion"]}")

with the respective versions

tealiumCoreVersion = "1.5.5" tealiumLifecycleVersion = "1.2.0" tealiumDispatchersVersion = "1.3.1" tealiumCollectDispatcherVersion = "1.1.1" tealiumAdIdentifierVersion = "1.1.1" tealiumAutoTrackingVersion = "1.0.1" tealiumTagManagementVersion = "1.2.1"

Any idea on how to troubleshoot this?

Thank you in advance :)

[jameskeith]

@gsarris Thanks for taking the time to raise this issue with us - sorry to hear you've been getting these crashes.

I've had a good look at the stack trace as well as our usage of the WebView and I'm not confident that we are going to be able to resolve this as an issue - I don't believe we're doing anything unusual.

The stack trace indicates that the cause is a ConcurrentModificationException thrown by a call to PowerManager.addThermalStatusListener - we don't make this call, and the stack trace suggests it comes from a JNI call made by the Chromium implementation.

We do make a reasonable effort to catch any Exceptions reported when instantiating the WebView, as it is notorious for being quite unreliable (sometimes missing) on various android systems.

As such, I would suggest that the error lies somewhere in a recent Chromium release; perhaps they only recently started setting a thermal status listener. I am not aware of any way to stop it registering a thermal listener, so suggest this will need to be raised with the Chromium Android team.

[gsarris]

@jameskeith Thx for the super fast reply :)

I went through the Chromium issue trackers and found these tealium-related stack traces.

Identical crash with PowerManager.addThermalStatusListener : https://issues.chromium.org/issues/324803204#comment22

Similar crash: https://issues.chromium.org/issues/324803204#comment28

The issue is marked as (Won't fix (Not reproducible)) unfortunately.

Any other hints for us? This is currently our number 1 crash...

Thx!

[jameskeith]

@gsarris Not a problem - I'm not ruling out that perhaps there is something else that we do that might make this a more likely scenario. However I'm stuck on two problems

• the error originates from us calling the WebView constructor - there is no other way for us to call this
• the exception is not thrown to our catch block, but instead to the thread's unhandled exception handler, which we therefore cannot catch/handle

I suspect that the issue you have linked was marked that way as those later stack traces do not match the cause that was originally raised (missing resources) I still believe that this should be an issue raised with the Google team - the Thermal Service registration shouldn't be throwing that exception as it's avoidable, and according to this very similar issue, I'm not sure the Chromium team are likely to add exception handling for this (comment 20)

If you'd like to raise a support ticket then perhaps we could assist in trying to track down some patterns that may highlight if there is something that can mitigate the problem in another way. Because at the moment, the only way to stop it from crashing is to stop creating a WebView which is a prerequisite of the TagManagement dispatcher