Uncaught BPMN Error leads to NullPointerException in OutputMapping on External Task

This issue was imported from JIRA:

Field	Value
JIRA Link	CAM-13325
Reporter	@tmetzke
Has restricted visibility comments	true

Environment (Required on creation):

Any Camunda distro on 7.15.0-alpha5
Flag skipOutputMappingOnCanceledActivities is set to true
Identified in the context of the RPA Bridge but applicable to External Tasks in general (possibly even to all activities)

Description (Required on creation; please attach any relevant screenshots, stacktraces, log files, etc. to the ticket):

Throwing a BPMN Error on an activity with an OutputMapping that is not caught (e.g. by a boundary event) leads to a NullPointerException in the OutputParameter because of no outer scope being present

java.lang.NullPointerException: null
    at org.camunda.bpm.engine.impl.core.variable.mapping.OutputParameter.execute(OutputParameter.java:52) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.core.variable.mapping.IoParameter.execute(IoParameter.java:51) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.core.variable.mapping.IoMapping.executeOutputParameters(IoMapping.java:45) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.destroy(ExecutionEntity.java:525) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.pvm.runtime.operation.PvmAtomicOperationProcessEnd.eventNotificationsCompleted(PvmAtomicOperationProcessEnd.java:83) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.pvm.runtime.operation.PvmAtomicOperationProcessEnd.eventNotificationsCompleted(PvmAtomicOperationProcessEnd.java:33) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.core.operation.AbstractEventAtomicOperation.execute(AbstractEventAtomicOperation.java:66) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.AtomicOperationInvocation.execute(AtomicOperationInvocation.java:99) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.invokeNext(CommandInvocationContext.java:131) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performNext(CommandInvocationContext.java:111) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performOperation(CommandInvocationContext.java:86) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performOperation(CommandInvocationContext.java:76) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:643) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:618) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.core.operation.AbstractEventAtomicOperation.execute(AbstractEventAtomicOperation.java:62) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.AtomicOperationInvocation.execute(AtomicOperationInvocation.java:99) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.invokeNext(CommandInvocationContext.java:131) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performNext(CommandInvocationContext.java:111) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performOperation(CommandInvocationContext.java:86) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performOperation(CommandInvocationContext.java:76) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:643) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:618) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.core.operation.AbstractEventAtomicOperation.execute(AbstractEventAtomicOperation.java:62) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.AtomicOperationInvocation.execute(AtomicOperationInvocation.java:99) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.invokeNext(CommandInvocationContext.java:131) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performNext(CommandInvocationContext.java:111) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performOperation(CommandInvocationContext.java:86) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:634) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:608) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.pvm.runtime.operation.PvmAtomicOperationActivityEnd.execute(PvmAtomicOperationActivityEnd.java:87) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.pvm.runtime.operation.PvmAtomicOperationActivityEnd.execute(PvmAtomicOperationActivityEnd.java:35) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.AtomicOperationInvocation.execute(AtomicOperationInvocation.java:99) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.invokeNext(CommandInvocationContext.java:131) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performNext(CommandInvocationContext.java:111) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performOperation(CommandInvocationContext.java:86) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandInvocationContext.performOperation(CommandInvocationContext.java:76) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.jobexecutor.AsyncContinuationJobHandler.execute(AsyncContinuationJobHandler.java:81) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.jobexecutor.AsyncContinuationJobHandler.execute(AsyncContinuationJobHandler.java:40) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.persistence.entity.JobEntity.execute(JobEntity.java:134) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.cmd.ExecuteJobsCmd.execute(ExecuteJobsCmd.java:110) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.cmd.ExecuteJobsCmd.execute(ExecuteJobsCmd.java:43) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:110) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:72) </sub><camunda-engine-spring-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) <sub><spring-tx-5.3.4.jar!/:5.3.4>
    at org.camunda.bpm.engine.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:70) </sub><camunda-engine-spring-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.CommandCounterInterceptor.execute(CommandCounterInterceptor.java:35) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobHelper.executeJob(ExecuteJobHelper.java:57) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.executeJob(ExecuteJobsRunnable.java:110) <sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at org.camunda.bpm.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:71) </sub><camunda-engine-7.15.0-alpha5-ee.jar!/:7.15.0-alpha5-ee>
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) <sub><na:na>
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) </sub><na:na>
    at java.base/java.lang.Thread.run(Thread.java:834) ~<na:na>

Steps to reproduce (Required on creation):

Create an activity with an OutputMapping (can be one that always succeeds and does not rely on variables or anything else)
Throw a BPMN error while on that activity (e.g. via API for an external task or Java Delegate or Script)

Observed Behavior (Required on creation):

A NPE is thrown

Expected behavior (Required on creation):

No NPE is thrown (either because the mapping is skipped or the behavior of the uncaught BPMN Error handling is changed)

Root Cause (Required on prioritization):

The uncaught BPMN Error leads to a simulated None End Event behavior which does technically not cancel the activity, which is why for example the mentioned flag to skip mappings is not considered
The activity as well as its parents are destroyed one after the other in ExecutionEntity#destroy - when the process instance is destroyed. The output parameter is executed twice: once when scope execution of the activity with the mapping is destroyed (=> expected) and once when the process instance execution is destroyed (=> not expected)
PvmAtomicOperationActivityEnd sets the activity on the next higher execution after removal (https://github.com/camunda/camunda-bpm-platform/blob/7.14.0/engine/src/main/java/org/camunda/bpm/engine/impl/pvm/runtime/operation/PvmAtomicOperationActivityEnd.java#L66)

Solution Ideas (Optional):

Hints (Optional):

Asynchronous continuation probably also needs to be considered here in case flag evaluation is expanded to cover this, as the None End Event behavior will be executed in the async continuation job context rather than in the execution context of throwing the BPMN Error itself and triggering that None End Event

Links:

is related to https://jira.camunda.com/browse/CAM-12704

This comment was imported from JIRA and written by user @ThorbenLindhauer

An idea for a fix could be to set the activity to null before calling process end. I pushed this to the branch: https://github.com/camunda/camunda-bpm-platform/commit/14e85aa5df05516673d10d8ee9294c16c3f67813

Then 25 test cases fail:

The state of the historic process instance depends on the presence of an activity => can probably be implemented in a different way
Some tests with execution listeners fail where apparently during the execution of an end listener the activity is already not set anymore (=> I did not fully understand this yet; we have to check what we guarantee here: it's clear that end listeners on the finishing activity must be able to access the activity from the DelegateExecution. For end listeners on the process level, we should check if we guarantee the same)

Full list of failures:

<ERROR> Failures:
<ERROR>   HistoryServiceTest.testHistoricProcessInstanceUserIdAndActivityId:212 expected:<theEnd> but was:<null>
<ERROR>   ProcessInstanceModificationSubProcessTest.shouldCompleteParentProcessWithParallelGateway:240
Expected: is <0L>
     but: was <1L>
<ERROR>   ProcessInstanceModificationSubProcessTest.shouldContinueParentProcess:192
Expected: is <1L>
     but: was <0L>
<ERROR>   ProcessInstanceModificationSubProcessTest.shouldContinueParentProcessWithMultiInstance:369
Expected: is <1L>
     but: was <0L>
<ERROR>   ProcessInstanceModificationSubProcessTest.shouldContinueParentProcessWithMultiInstanceEmbeddedSubProcess:562
Expected: is <1L>
     but: was <0L>
<ERROR>   ProcessInstanceModificationSubProcessTest.shouldContinueParentProcessWithMultiInstanceInsideEmbeddedSubProcess:466
Expected: is <1L>
     but: was <0L>
<ERROR>   ProcessInstantiationAtActivitiesTest.testStartMultipleTasksInSyncProcess:537 expected:<end> but was:<null>
<ERROR>   BoundaryErrorEventTest.testDeeplyNestedErrorThrownOnlyAutomaticSteps:223 expected:<processEnd1> but was:<null>
<ERROR>   ExecutionListenerBpmnModelExecutionContextTest.testProcessEndEvent:160->assertFlowElementIs:181
<ERROR>   ExecutionListenerTest.testMultiInstanceCancelationDoesNotAffectEndListener:508
<ERROR>   ExecutionListenerTest.testScriptListener:284 Unable ot find variable with name 'end-end'
<ERROR>   ExecutionListenerTest.testScriptResourceListener:308 Unable ot find variable with name 'end-end'
<ERROR>   HistoricProcessInstanceStateTest.testCompletedOnEndEvent:86 expected:<"<COMPLE>TED"> but was:<"<INTERNALLY_TERMINA>TED">
<ERROR>   HistoricProcessInstanceStateTest.testCompletionWithSuspension:123 expected:<"<COMPLE>TED"> but was:<"<INTERNALLY_TERMINA>TED">
<ERROR>   HistoricProcessInstanceStateTest.testErrorEndEvent:231 expected:<"<COMPLE>TED"> but was:<"<INTERNALLY_TERMINA>TED">
<ERROR>   HistoricProcessInstanceStateTest.testTerminatedInternalWithGateway:74 expected:<"<COMPLE>TED"> but was:<"<INTERNALLY_TERMINA>TED">
<ERROR>   HistoricProcessInstanceStateTest.testWithCallActivity:249 expected:<"<COMPLE>TED"> but was:<"<INTERNALLY_TERMINA>TED">
<ERROR>   SkipOutputMappingOnCanceledActitivitesTest.shouldSkipOutputMappingOnBpmnErrorAtExternalTasWithUncaughtErrorAsyncAfter:138 expected:<"<COMPLE>TED"> but was:<"<INTERNALLY_TERMINA>TED">
<ERROR>   SkipOutputMappingOnCanceledActitivitesTest.shouldSkipOutputMappingOnBpmnErrorAtExternalTaskWithUncaughtError:102 expected:<"<COMPLE>TED"> but was:<"<INTERNALLY_TERMINA>TED">
<ERROR>   ProcessDataLoggingContextTest.shouldLogCustomMdcPropertiesOnlyInActivityContext:186->assertActivityLogsPresent:683->assertLogs:732
<ERROR>   ProcessDataLoggingContextTest.shouldLogMdcPropertiesForAsyncAfterInTaskContext:254->assertActivityLogsPresent:667->assertActivityLogs:676->assertActivityLogs:707->assertLogs:732
<ERROR>   ProcessDataLoggingContextTest.shouldLogMdcPropertiesForAsyncBeforeInTaskContext:237->assertActivityLogsPresent:667->assertActivityLogs:676->assertActivityLogs:707->assertLogs:732
<ERROR>   ProcessDataLoggingContextTest.shouldLogMdcPropertiesForTimerInTaskContext:273->assertActivityLogsPresent:667->assertActivityLogs:676->assertActivityLogs:707->assertLogs:732
<ERROR>   ProcessDataLoggingContextTest.shouldLogMdcPropertiesOnlyInActivityContext:165->assertActivityLogsPresent:667->assertActivityLogs:676->assertActivityLogs:707->assertLogs:732
<ERROR>   ProcessDataLoggingContextTest.shouldNotLogBusinessKeyIfNotConfigured:137->assertActivityLogs:676->assertActivityLogs:707->assertLogs:732
<ERROR> Errors:
<ERROR>   ProcessInstanceModificationSubProcessTest.shouldContinueParentProcessWithParallelGateway:290 NullPointer
<ERROR>   TaskListenerDelegateCompletionTest.testCompletionIsPossibleAfterAssignmentUpdate:123 ▒ ProcessEngine
<ERROR>   PvmActivityInstanceTest.testStartInSubProcess:526 ▒ NullPointer
<ERROR>   PvmActivityInstanceTest.testSubProcessNoEnd:471 ▒ NullPointer
<INFO>
<ERROR> Tests run: 14821, Failures: 25, Errors: 4, Skipped: 86

camunda / camunda-bpm-platform

Uncaught BPMN Error leads to NullPointerException in OutputMapping on External Task #2643