camunda / camunda

Process Orchestration Framework
https://camunda.com/platform/
3.34k stars 610 forks source link

NPE when recovering from snapshot in StreamProcessor #12201

Open npepinpe opened 1 year ago

npepinpe commented 1 year ago

Describe the bug

In one case with 8.2.0-rc1, a stream processor instance failed in STARTED phase while trying to recover from snapshot, because its logStreamReader instance was null. Looking at the logs, we can see previously that it had failed to retrieve a logStreamReader instance in its STARTING phase.

We would expect the actor to have failed then, in the starting phase, but it seems it kept going into the STARTED phase, without a reader, resulting in the NPE.

See https://console.cloud.google.com/errors/detail/CJ60wpTCkMbM4AE;service=zeebe;version=release-8-2-0-rc1?project=zeebe-io

Expected behavior

No NPE occurs, and we fail the actor in the starting phase instead of moving on to the started phase when we're unable to get a log stream reader.

Log/Stacktrace

Full Stacktrace

``` ava.lang.NullPointerException: Cannot invoke "io.camunda.zeebe.logstreams.log.LogStreamReader.seekToNextEvent(long)" because "this.logStreamReader" is null at io.camunda.zeebe.stream.impl.StreamProcessor.recoverFromSnapshot(StreamProcessor.java:389) ~[zeebe-stream-platform-8.2.0-SNAPSHOT.jar:8.2.0-SNAPSHOT] at io.camunda.zeebe.stream.impl.StreamProcessor.onActorStarted(StreamProcessor.java:161) ~[zeebe-stream-platform-8.2.0-SNAPSHOT.jar:8.2.0-SNAPSHOT] at io.camunda.zeebe.scheduler.ActorJob.invoke(ActorJob.java:92) [zeebe-scheduler-8.2.0-SNAPSHOT.jar:8.2.0-SNAPSHOT] at io.camunda.zeebe.scheduler.ActorJob.execute(ActorJob.java:45) [zeebe-scheduler-8.2.0-SNAPSHOT.jar:8.2.0-SNAPSHOT] at io.camunda.zeebe.scheduler.ActorTask.execute(ActorTask.java:119) [zeebe-scheduler-8.2.0-SNAPSHOT.jar:8.2.0-SNAPSHOT] at io.camunda.zeebe.scheduler.ActorThread.executeCurrentTask(ActorThread.java:106) [zeebe-scheduler-8.2.0-SNAPSHOT.jar:8.2.0-SNAPSHOT] at io.camunda.zeebe.scheduler.ActorThread.doWork(ActorThread.java:87) [zeebe-scheduler-8.2.0-SNAPSHOT.jar:8.2.0-SNAPSHOT] at io.camunda.zeebe.scheduler.ActorThread.run(ActorThread.java:198) [zeebe-scheduler-8.2.0-SNAPSHOT.jar:8.2.0-SNAPSHOT] ```

Environment:

megglos commented 1 year ago

ZDP Triage: