search

patrykandpatrick / vico

A light and extensible chart library for Android.
https://patrykandpatrick.com/vico/wiki
Apache License 2.0
2.13k stars 130 forks source link

Couple crashes with `java.util.ConcurrentModificationException` and `java.util.NoSuchElementException: Key null is missing in the map` #736

Open L-Andrade opened 5 months ago

L-Andrade commented 5 months ago

How to reproduce

Hey! Thanks again for the library. It's been working great so far, but we've started getting a couple crashes now after pushing it to more users (nothing major yet).

I've tried to reproduce it for the last couple days, but haven't managed to.

I have a LazyColumn with a CartesianChartHost. This is the basic gist of the chart:

ProvideVicoTheme(rememberM2VicoTheme()) {
    val modelProducer = remember { CartesianChartModelProducer.build() }
    LaunchedEffect(entries) {
        modelProducer.tryRunTransaction {
            lineSeries { series(x = entries.map { it.key }, y = entries.map { it.amount }) }
        }
    }
    // ...
    CartesianChartHost(
            modifier = modifier,
            scrollState = rememberVicoScrollState(scrollEnabled = false),
            modelProducer = modelProducer,
            diffAnimationSpec = spring(stiffness = Spring.StiffnessVeryLow),
            chart = rememberCartesianChart(
                rememberLineCartesianLayer(
                    lines = listOf(rememberLineSpec(shader = shader, backgroundShader = null)),
                    axisValueOverrider = remember {
                        object : AxisValueOverrider {
                            override fun getMinY(minY: Float, maxY: Float, extraStore: ExtraStore): Float {
                                return (minY - 0.0005f).coerceAtLeast(0f)
                            }

                            override fun getMaxY(minY: Float, maxY: Float, extraStore: ExtraStore): Float {
                                return (maxY + 0.0005f).coerceAtLeast(0.1f)
                            }
                        }
                    },
                ),
                decorations = listOf(
                    rememberHorizontalLine(
                        y = { state.start },
                        line = rememberLineComponent(
                            color = MaterialTheme.colors.gray80, shape = remember { DashedShape() },
                        ),
                    ),
                ),
                persistentMarkers = // Custom logic to display some markers
            ),
            marker = primaryMarker,
            markerVisibilityListener = // Custom logic for the marker visibility
        )
}

I have removed a few bits for brevity and because I can not share the whole code. Let me know if any of it is important to figuring this issue.

The chart is only shown if there are entries to display (entries is not empty). It seems to happen after the user scrolls to the bottom of the screen (chart is at the top).

Could it be because the CartesianChartModelProducer is recreated when the chart is visible again? Since it's in a LazyColumn, the chart leaves the composition and then returns once the user sees it again. Not sure it should matter though, since the chart is also a "new" one.

I'm not entirely sure if it is an issue on our end or an issue in Vico's end. Do you folks have any idea of how we can try to replicate this issue?

Let me know if you see any issue in our implementation too, please!

Observed behavior

There are a few crashes. I have a couple stack traces, not sure if they're linked to the same issue or not

      Fatal Exception: java.util.ConcurrentModificationException:
       at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:760)
       at java.util.LinkedHashMap$LinkedValueIterator.next(LinkedHashMap.java:787)
       at kotlin.collections.CollectionsKt___CollectionsKt.mapTo(CollectionsKt___Collections.kt:1620)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer.tryUpdate(CartesianChartModelProducer.kt:59)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer.access$tryUpdate(CartesianChartModelProducer.kt:38)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer$Transaction.tryCommit(CartesianChartModelProducer.kt:206)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer.tryRunTransaction(CartesianChartModelProducer.kt:171)
       at com.getbux.android.stocks.ui.chart.ValueChartKt$ValueChart$1$1.invokeSuspend(ValueChart.kt:55)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
       at androidx.compose.ui.platform.AndroidUiDispatcher.performTrampolineDispatch(AndroidUiDispatcher.android.kt:81)
       at androidx.compose.ui.platform.AndroidUiDispatcher.access$performTrampolineDispatch(AndroidUiDispatcher.android.kt:41)
       at androidx.compose.ui.platform.AndroidUiDispatcher$dispatchCallback$1.run(AndroidUiDispatcher.android.kt:57)
       at android.os.Handler.handleCallback(Handler.java:958)
       at android.os.Handler.dispatchMessage(Handler.java:99)
       at android.os.Looper.loopOnce(Looper.java:224)
       at android.os.Looper.loop(Looper.java:318)
       at android.app.ActivityThread.main(ActivityThread.java:8677)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:561)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1013)

and

      Fatal Exception: java.util.NoSuchElementException: Key null is missing in the map.
       at kotlin.collections.MapsKt__MapWithDefaultKt.getOrImplicitDefaultNullable(MapsKt__MapWithDefault.kt:24)
       at kotlin.collections.MapsKt__MapsKt.getValue(MapsKt__Maps.kt:360)
       at com.patrykandpatrick.vico.core.cartesian.data.MutableChartValuesKt$toImmutable$1.getYRange(MutableChartValues.kt:122)
       at com.patrykandpatrick.vico.core.cartesian.layer.LineCartesianLayer.toDrawingModel(LineCartesianLayer.kt:598)
       at com.patrykandpatrick.vico.core.cartesian.layer.LineCartesianLayer.prepareForTransformation(LineCartesianLayer.kt:583)
       at com.patrykandpatrick.vico.core.cartesian.layer.LineCartesianLayer.prepareForTransformation(LineCartesianLayer.kt:73)
       at com.patrykandpatrick.vico.core.cartesian.CartesianChart$transformationPreparationModelAndLayerConsumer$1.invoke(CartesianChart.java:106)
       at com.patrykandpatrick.vico.core.cartesian.CartesianChart.consume(CartesianChart.kt:328)
       at com.patrykandpatrick.vico.core.cartesian.CartesianChart.forEachWithLayer(CartesianChart.kt:316)
       at com.patrykandpatrick.vico.core.cartesian.CartesianChart.prepareForTransformation(CartesianChart.kt:251)
       at com.patrykandpatrick.vico.compose.cartesian.CartesianChartModelProducerKt$collectAsState$2$1$2.invoke(CartesianChartModelProducer.kt:124)
       at com.patrykandpatrick.vico.compose.cartesian.CartesianChartModelProducerKt$collectAsState$2$1$2.invoke(CartesianChartModelProducer.kt:124)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer$UpdateReceiver.handleUpdate(CartesianChartModelProducer.kt:229)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer.registerForUpdates(CartesianChartModelProducer.kt:146)
       at com.patrykandpatrick.vico.compose.cartesian.CartesianChartModelProducerKt$collectAsState$2$1.invokeSuspend(CartesianChartModelProducer.kt:114)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:585)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693)

Expected behavior

The app should not crash

Vico version(s)

2.0.0-alpha.20

Android version(s)

12, 13, 14

Additional information

We're actually on 2.0.0-alpha.19, but there seem to be only small changes in alpha.20.

It's quite hard to figure out where the issue is coming from. Since the stack trace does not point to our app code at all, we don't know if it's an issue with our integration or an internal Vico issue.

And since we're not familiar with Vico's codebase, it's a bit harder to try to replicate it. I've tried scrolling up and down multiple times, adding delays in multiple places, adding more charts to the screen and scrolling, configuration changes, going to the background/foreground, spamming buttons that cause the entries to change, navigating back or to another screen, and more. Let me know if you have any ideas that could replicate it, any help would be great here!

Thanks in advance

Gowsky commented 5 months ago

Hello. I did some stress testing and I haven't experienced this crash.

Could it be because the CartesianChartModelProducer is recreated when the chart is visible again?

This shouldn't be the reason. Have you tried putting CartesianChartModelProducer in the viewmodel, as suggested here? The examples use LaunchedEffect for simplicity, but especially when the chart leaves and enters the composition, it's more optimal to store the producer in the viewmodel.

Do you folks have any idea of how we can try to replicate this issue?

It's hard to tell without specific information. Is there anything common between the incidents? Like low-spec devices?

Let me know if you see any issue in our implementation too, please!

I can't see any issues that could cause this.

ibcurly commented 5 months ago

I also got this crash, spent a few hours looking for the culprit no luck yet

in the below code, yRanges[axisPosition] returns null because yRanges is empty and yRanges.getValue(null) throws the NoSuchElementException with the message "Key null is missing in the map."

in MutableChartValues.kt

public fun MutableChartValues.toImmutable(): ChartValues =
    object : ChartValues {
        private val yRanges = this@toImmutable.yRanges
        override val minX: Float = this@toImmutable.minX
        override val maxX: Float = this@toImmutable.maxX
        override val xStep: Float = this@toImmutable.xStep
        override val model: CartesianChartModel = this@toImmutable.model.toImmutable()
        override fun getYRange(axisPosition: AxisPosition.Vertical?): ChartValues.YRange =
            yRanges[axisPosition] ?: yRanges.getValue(null)
    }

UPDATE:

I was using the chart host that takes in a CartesianChartModel instead of the CartesianChartModelProducer. After changing it to match OP's implementation with the model producer and the LaunchedEffect, I no longer see the issue, so I'm not sure why OP has the issue. 🤷

L-Andrade commented 5 months ago

Hey! Thanks for the reply

Hello. I did some stress testing and I haven't experienced this crash.

Could it be because the CartesianChartModelProducer is recreated when the chart is visible again?

This shouldn't be the reason. Have you tried putting CartesianChartModelProducer in the viewmodel, as suggested here? The examples use LaunchedEffect for simplicity, but especially when the chart leaves and enters the composition, it's more optimal to store the producer in the viewmodel.

Yes, looking further into it, it seems like putting it in the ViewModel would most likely solve it. We were just trying to avoid it since we wanted the chart implementation to be part of a UI-only module, and this way we'll also have to put it a Vico dependency in the module that the ViewModel is in. Or we will have to create a wrapper around it. Reason we wanted to keep it out of the ViewModel is in case we'd want to change the implementation or library, we would have fewer places to change and it would be more contained.

Do you folks have any idea of how we can try to replicate this issue?

It's hard to tell without specific information. Is there anything common between the incidents? Like low-spec devices?

So far only had issues with Android 12, 13, and 14. Devices are not low-spec, they're decent (most common so far is Galaxy A53 5G).

I tried to understand the internal Vico code and had an idea, but might be totally wrong. Maybe still worth checking on your side just in case though. Click here to check the details I was taking a look at the logic in `CartesianChartModelProducer`, and I think there are potential race conditions. With debugging breakpoints (non-suspend, log only) we can see that `registerForUpdates` (suspend function) and `unregisterFromUpdates` can be called before, during, or after `tryUpdate`. This can mean that a transaction might be running when a receiver is registered/unregistered, which could cause the `ConcurrentModificationException`. `registerForUpdates` is also called inside a `scope.launch` - which is scheduled to be immediately run, but might there might be a delay (albeit small of course) if To solve it, registering or unregistering a receiver would need to either: 1. Check and wait for the `Mutex` lock before setting/removing 2. Cancel any ongoing transaction before setting/removing Otherwise there might be an unfortunate timing.

With that said, I still can't replicate the issue, so I'm probably still missing something! I think I will try to move it to the ViewModel and see if it's fixed, but I believe the race condition will still be possible, just less probable (almost impossible, I guess) since the transaction will most likely be done before the chart is composed, and as such the receiver will always be registered after the transaction is complete (and fewer transactions will also be done, since it will not be done every time the chart is composed), and there's a smaller chance of a transaction being ran when unregistering since there will be fewer transactions too.

Gowsky commented 3 months ago

@L-Andrade, Vico 2.0.0-alpha.22 fixes the registration problem you described. This should fix the ConcurrentModificationException. As the release notes describe, you must switch to runTransaction to benefit from the fix. tryRunTransaction is now deprecated. NoSuchElementException might be fixed as well. We can't be sure, as we're lacking the reproduction. Please let us know if you still experience it.

@ibcurly, thanks for reporting it. Unfortunately, we weren't able to reproduce it, so we'll need a MRE to fix this.

L-Andrade commented 3 months ago

Hey @Gowsky! Thanks for the update. I took a look and it looks like a nice improvement. In the meantime, we had already shipped the recommended fix of putting the transaction call in the ViewModel (we abstracted it away in a class) which seemed to fix the issue, and it makes sense since it causes much fewer transactions.

However, I saw the recommendation in the Vico docs asking to change the dispatcher to Default on the caller side. I believe this should be something done in the Vico side, following the guideline that suspend functions should be main safe from the caller:

Other big libraries also respect this guideline, such as Room and Retrofit. This would be a further improvement, a bit less error-prone for the library users

Thanks again!

patrickmichalik commented 3 months ago

Thanks for the update and the suggestion, @L-Andrade! Vico 2.0.0 Alpha 23 makes CartesianChartModelProducer’s suspending functions main-safe. Execution is now moved to Dispatchers.Default internally.

While, as mentioned by @Gowsky, we haven’t been able to reproduce the NoSuchElementException, we believe it stemmed from the same problem as the ConcurrentModificationException. With this problem having been resolved, neither of the two reported exceptions should occur anymore. I’ll thus be adding the “done in alpha” label. Should you face any issues, please let us know. Cheers!

L-Andrade commented 3 months ago

Thanks both! I also believe the changes fix the issue. We will update to that version when we can.

Feel free to close the issue!

Gowsky commented 3 months ago

Since Vico 1 is still supported, we keep bug reports open until they're addressed in both Vico versions. The "Done in alpha" label basically means that the issue is closed as far as Vico 2 is concerned.

mak1nt0sh commented 6 days ago

Just tried beta.2 and the crash reappeared:

java.util.NoSuchElementException: Key null is missing in the map.
    at kotlin.collections.MapsKt__MapWithDefaultKt.getOrImplicitDefaultNullable(MapWithDefault.kt:24)
    at kotlin.collections.MapsKt__MapsKt.getValue(Maps.kt:369)
    at com.patrykandpatrick.vico.core.cartesian.data.MutableCartesianChartRangesKt$toImmutable$1.getYRange(MutableCartesianChartRanges.kt:99)
    at com.patrykandpatrick.vico.core.cartesian.axis.DefaultVerticalAxisItemPlacer.getHeightMeasurementLabelValues(DefaultVerticalAxisItemPlacer.kt:56)
    at com.patrykandpatrick.vico.core.cartesian.axis.VerticalAxis.getMaxLabelHeight(VerticalAxis.kt:362)
    at com.patrykandpatrick.vico.core.cartesian.axis.VerticalAxis.updateInsets(VerticalAxis.kt:305)
    at com.patrykandpatrick.vico.core.cartesian.axis.VerticalAxis.updateInsets(VerticalAxis.kt:58)
    at com.patrykandpatrick.vico.core.cartesian.CartesianChart.prepare(CartesianChart.kt:229)
    at com.patrykandpatrick.vico.compose.cartesian.CartesianChartHostKt.CartesianChartHostImpl$lambda$23$lambda$22(CartesianChartHost.kt:195)
    at com.patrykandpatrick.vico.compose.cartesian.CartesianChartHostKt.$r8$lambda$aQQKwZOn9huYyaTRo_AvgGCegn8(Unknown Source:0)
    at com.patrykandpatrick.vico.compose.cartesian.CartesianChartHostKt$$ExternalSyntheticLambda5.invoke(D8$$SyntheticClass:0)
    at androidx.compose.ui.draw.DrawBackgroundModifier.draw(DrawModifier.kt:127)
    at androidx.compose.ui.node.LayoutNodeDrawScope.drawDirect-eZhPAX0$ui_release(LayoutNodeDrawScope.kt:110)
    at androidx.compose.ui.node.LayoutNodeDrawScope.draw-eZhPAX0$ui_release(LayoutNodeDrawScope.kt:89)
    at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:450)
    at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
    at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
    at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
    at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
    at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
    at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
    at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
    at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
    at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
    at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
    at androidx.compose.ui.node.LayoutNode.draw$ui_release(LayoutNode.kt:1000)
    at androidx.compose.ui.node.InnerNodeCoordinator.performDraw(InnerNodeCoordinator.kt:196)
    at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
    at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
    at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
    at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
    at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
    at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
    at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
    at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
    at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
    at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
    at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
    at androidx.compose.ui.node.LayoutNode.draw$ui_release(LayoutNode.kt:1000)
    at androidx.compose.ui.node.InnerNodeCoordinator.performDraw(InnerNodeCoordinator.kt:196)
Gowsky commented 6 days ago

Hello, @mak1nt0sh. Can you share more details and some context for the issue? Can you provide an MRE?