[BUG] Variables aren't restored from persistence for Blocking Activities - RunTask

NikiforovAll commented 3 weeks ago

Description

It seems like the execution context is not restored after a blocking activity is triggered. I've checked the snapshot of a workflow instance and it seems like dynamic variables are not assigned correctly.

Steps to Reproduce

Start a worfklow that contains ParallelForEach and RunTask activity in it.
Once we try to submit a task result via /elsa/api/tasks/{{taskId}}/complete we receive a Null Reference Exception (NRE) because the context is not restored and variable is no longer assigned.

public class QuorumVotingWorkflow : WorkflowBase
{
    protected override void Build(IWorkflowBuilder builder)
    {
        builder.Version = 1;

        var board = builder.WithVariable<VotingBoard>();

        var currentVoteResult = builder.WithVariable<bool>();

        var votes = builder.WithVariable<Dictionary<string, bool>>();

        var approved = builder.WithVariable<bool>();

        builder.Root = new Sequence
        {
            Activities =
            {
                new HttpEndpoint
                {
                    Path = new("/parallel-quorum-vote"),
                    CanStartWorkflow = true,
                    SupportedMethods = new([HttpMethods.Post]),
                    ParsedContent = new(board)
                },
                new WriteHttpResponse
                {
                    ContentType = new(MediaTypeNames.Application.Json),
                    Content = new(board.Get),
                },
                new ParallelForEach<User>()
                {
                    // doesn't work - Null Reference Excetion is thrown after resolution of any of `RunTask`
                    Items = new(context => board.Get(context).Members),
                    Body = new Sequence
                    {
                        Activities =
                        {
                            new RunTask("PromptVote")
                            {
                                Result = new Output<object>(currentVoteResult),
                                Payload = new(context => new Dictionary<string, object>
                                {
                                    ["User"] = context.GetVariable<User>("CurrentValue"),
                                })
                            },
                            new SetVariable<Dictionary<string, bool>>(
                                votes,
                                WorkflowExtensions.SetVotes(currentVoteResult, votes)
                            )
                        }
                    },
                },
                new QuorumVote()
                {
                    Votes = new(context => votes.Get(context).Values),
                    Result = new(approved),
                },
                new If(approved.Get<bool>)
                {
                    Then = new WriteLine(context => $"Approved!"),
                    Else = new WriteLine(context => $"Rejected!")
                }
            },
        };
    }
}

I've documented the result of the investigation as part of this project: https://github.com/NikiforovAll/elsa-workflows-voting

Attachments:

https://github.com/NikiforovAll/elsa-workflows-voting

Reproduction Rate: Every time
Video/Screenshots:
Additional Configuration:

Expected Behavior

I expect workflow to continue working after a blocking activity is resolved

Actual Behavior

The context is missing

Screenshots

Environment

Log Output

Troubleshooting Attempts

I've tried various workflows implementations that will suffice my requirement. But the principle remains the same - context is not restored after blocking activities.

Additional Context

glime-ai[bot] commented 3 weeks ago

Based on the information provided, it appears that you are encountering an issue with the Elsa Workflow framework where the execution context is not properly restored after a blocking activity, such as ParallelForEach and RunTask, is triggered. This results in a Null Reference Exception when trying to complete a task via the /elsa/api/tasks/{{taskId}}/complete endpoint.

This issue seems to be related to the deserialization of workflow variables after a workflow has been suspended and then resumed. The problem might be due to the way variables are handled or a bug within the Elsa framework itself.

To address this issue, you may want to:

Ensure that you are using the latest stable version of Elsa, as bugs are regularly fixed in new releases.
Review the Elsa documentation and GitHub issues for any similar reported problems and potential workarounds or fixes.
Consider opening a new issue on the Elsa GitHub repository if you believe you have encountered a bug that has not been reported or resolved yet. Provide as much detail as possible, including steps to reproduce, expected behavior, actual behavior, and any exception messages or stack traces.

Additionally, you can explore the following resources for more information and potential solutions:

Elsa GitHub Issues: https://github.com/elsa-workflows/elsa-core/issues
Elsa Documentation: https://elsa-workflows.github.io/elsa-core/

If you have already tried the above steps or if you have additional information that could help in troubleshooting, please provide that information to further assist you.

This comment was generated by Glime.

NikiforovAll commented 2 weeks ago

@sfmskywalker Could you please take a look?

sfmskywalker commented 2 weeks ago

Hi @NikiforovAll , thank you for the detailed repro. Variables by default are transitory and not persisted. To do so, you need to configure the variables to use a storage driver. For example:

var board = builder.WithVariable<VotingBoard>().WithWorkflowStorage();

var currentVoteResult = builder.WithVariable<bool>().WithWorkflowStorage();

var votes = builder.WithVariable<Dictionary<string, bool>>().WithWorkflowStorage();

var approved = builder.WithVariable<bool>().WithWorkflowStorage();

When designing workflows using Elsa Studio, workflow variables are automatically configured to use the Workflow storage provider. But when creating workflows from code, you need to explicitly configure what variables should be persisted across bursts of execution.

NikiforovAll commented 2 weeks ago

@sfmskywalker

Yes, thank you for the reply.

I've already tried this approach and I can see it in PersistentVariablesDictionary . But, unfortunately, the bug remains. The workflow fails after the first vote (replay from persistence)

Commit: https://github.com/NikiforovAll/elsa-workflows-voting/commit/5cfa72882230b5f223de17d246e0f246de7f58d1

Steps to reproduce:

###
POST {{url}}/workflows/parallel-quorum-vote
Content-Type: application/json

{
    "members": [
        {"name": "User 1", "id": "1"},
        {"name": "User 2", "id": "2"}
    ]
}

###
@taskId=1befa6ede27e4b87
@apiKey=00000000-0000-0000-0000-000000000000

POST {{url}}/elsa/api/tasks/{{taskId}}/complete
Content-Type: application/json
Authorization: ApiKey {{apiKey}}

{
    "result": false
}

{
    "$id": "1",
    "id": "3a7183ac137db70f",
    "definitionId": "QuorumVotingWorkflow",
    "definitionVersionId": "cd7291eace9fe745",
    "definitionVersion": 1,
    "status": "Finished",
    "subStatus": "Faulted",
    "bookmarks": {
        "$id": "2",
        "$values": [
            {
                "$id": "3",
                "id": "abc1450b5b8bc0e0",
                "name": "Elsa.RunTask",
                "hash": "69C8F95FF69CE30146879B9FEFC6A0C9E436B9F2D42D274A05B8A5314C3FEB0E",
                "payload": {
                    "$id": "4",
                    "taskId": "4ff777efb8f211eb",
                    "taskName": "PromptVote",
                    "_type": "Elsa.Workflows.Runtime.Bookmarks.RunTaskBookmarkPayload, Elsa.Workflows.Runtime"
                },
                "activityId": "RunTask1",
                "activityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2:RunTask1",
                "activityInstanceId": "c56b45635a7d2ad0",
                "createdAt": "2024-04-24T17:50:20.4792127+00:00",
                "autoBurn": true,
                "callbackMethodName": "ResumeAsync",
                "autoComplete": true
            }
        ]
    },
    "incidents": {
        "$id": "5",
        "$values": [
            {
                "$id": "6",
                "activityId": "SetVariable\u003CDictionary\u003CString,Boolean\u003E\u003E1",
                "activityType": "Elsa.SetVariable\u003CDictionary\u003CString,Boolean\u003E\u003E",
                "message": "Object reference not set to an instance of an object.",
                "exception": {
                    "$id": "7",
                    "type": "System.NullReferenceException, System.Private.CoreLib",
                    "message": "Object reference not set to an instance of an object.",
                    "stackTrace": "   at Elsa.Demo.API.Workflows.WorkflowExtensions.\u003C\u003Ec__DisplayClass0_0.\u003CSetVotes\u003Eb__0(ExpressionExecutionContext context) in C:\\Users\\Oleksii_Nikiforov\\adgm\\workflow-engine\\src\\API\\Workflows\\WorkflowExtensions.cs:line 20\r\n   at Elsa.Expressions.Models.Expression.\u003C\u003Ec__DisplayClass14_0\u00601.\u003CDelegateExpression\u003Eb__0(ExpressionExecutionContext context)\r\n   at Elsa.Expressions.DelegateExpressionHandler.EvaluateAsync(Expression expression, Type returnType, ExpressionExecutionContext context, ExpressionEvaluatorOptions options)\r\n   at Elsa.Expressions.Services.ExpressionEvaluator.EvaluateAsync(Expression expression, Type returnType, ExpressionExecutionContext context, ExpressionEvaluatorOptions options)\r\n   at Elsa.Extensions.ActivityExecutionContextExtensions.EvaluateInputPropertyAsync(ActivityExecutionContext context, ActivityDescriptor activityDescriptor, InputDescriptor inputDescriptor)\r\n   at Elsa.Extensions.ActivityExecutionContextExtensions.EvaluateInputPropertiesAsync(ActivityExecutionContext context)\r\n   at Elsa.Workflows.Middleware.Activities.DefaultActivityInvokerMiddleware.EvaluateInputPropertiesAsync(ActivityExecutionContext context)\r\n   at Elsa.Workflows.Middleware.Activities.DefaultActivityInvokerMiddleware.InvokeAsync(ActivityExecutionContext context)\r\n   at Elsa.Workflows.Middleware.Activities.NotificationPublishingMiddleware.InvokeAsync(ActivityExecutionContext context)\r\n   at Elsa.Workflows.Middleware.Activities.ExecutionLogMiddleware.InvokeAsync(ActivityExecutionContext context)\r\n   at Elsa.Workflows.Middleware.Activities.ExceptionHandlingMiddleware.InvokeAsync(ActivityExecutionContext context)"
                },
                "timestamp": "2024-04-24T17:50:49.9710492+00:00"
            }
        ]
    },
    "isSystem": false,
    "completionCallbacks": {
        "$id": "8",
        "$values": [
            {
                "$id": "9",
                "ownerInstanceId": "6ef441ea2ff2f48a",
                "childNodeId": "Workflow1:Sequence1",
                "methodName": "OnRootCompletedAsync"
            },
            {
                "$id": "10",
                "ownerInstanceId": "d545156819159bca",
                "childNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1",
                "methodName": "OnChildCompleted"
            },
            {
                "$id": "11",
                "ownerInstanceId": "9f159acbf8ed8cf9",
                "childNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2",
                "methodName": "OnChildCompleted",
                "tag": "13a17d90-f440-49bd-a84b-0c6ef45d1d04"
            },
            {
                "$id": "12",
                "ownerInstanceId": "9f159acbf8ed8cf9",
                "childNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2",
                "methodName": "OnChildCompleted",
                "tag": "d53ff52c-bfa4-4bb4-aaff-d09f95986cf9"
            },
            {
                "$id": "13",
                "ownerInstanceId": "50eefe4e0e36361e",
                "childNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2:RunTask1",
                "methodName": "OnChildCompleted"
            },
            {
                "$id": "14",
                "ownerInstanceId": "8145154905ebf214",
                "childNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2:SetVariable\u003CDictionary\u003CString,Boolean\u003E\u003E1",
                "methodName": "OnChildCompleted"
            }
        ]
    },
    "activityExecutionContexts": {
        "$id": "15",
        "$values": [
            {
                "$id": "16",
                "id": "d4fd9b446c00b793",
                "parentContextId": "8145154905ebf214",
                "scheduledActivityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2:SetVariable\u003CDictionary\u003CString,Boolean\u003E\u003E1",
                "ownerActivityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2",
                "properties": {
                    "$id": "17"
                },
                "activityState": {
                    "$id": "18",
                    "Variable": {
                        "id": "Workflow1:variable-3",
                        "name": "Variable_2",
                        "typeName": "System.Collections.Generic.Dictionary\u00602[[System.String, System.Private.CoreLib],[System.Boolean, System.Private.CoreLib]], System.Private.CoreLib",
                        "storageDriverTypeName": "Elsa.Workflows.Services.WorkflowStorageDriver, Elsa.Workflows.Core"
                    }
                },
                "dynamicVariables": {
                    "$id": "19",
                    "$values": []
                },
                "status": "Faulted",
                "startedAt": "2024-04-24T17:50:49.9575421+00:00"
            },
            {
                "$id": "20",
                "id": "6ef441ea2ff2f48a",
                "scheduledActivityNodeId": "Workflow1",
                "properties": {
                    "$id": "21",
                    "PersistentVariablesDictionary": {
                        "Workflow1:variable-2": false,
                        "_type": "System.Collections.Generic.Dictionary\u00602[[System.String, System.Private.CoreLib],[System.Object, System.Private.CoreLib]], System.Private.CoreLib"
                    }
                },
                "activityState": {
                    "$id": "22"
                },
                "dynamicVariables": {
                    "$id": "23",
                    "$values": []
                },
                "status": "Running",
                "startedAt": "2024-04-24T17:50:20.3963146+00:00"
            },
            {
                "$id": "24",
                "id": "d545156819159bca",
                "parentContextId": "6ef441ea2ff2f48a",
                "scheduledActivityNodeId": "Workflow1:Sequence1",
                "ownerActivityNodeId": "Workflow1",
                "properties": {
                    "$id": "25",
                    "CurrentIndex": 3
                },
                "activityState": {
                    "$id": "26"
                },
                "dynamicVariables": {
                    "$id": "27",
                    "$values": []
                },
                "status": "Running",
                "startedAt": "2024-04-24T17:50:20.4061082+00:00"
            },
            {
                "$id": "28",
                "id": "9f159acbf8ed8cf9",
                "parentContextId": "d545156819159bca",
                "scheduledActivityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1",
                "ownerActivityNodeId": "Workflow1:Sequence1",
                "properties": {
                    "$id": "29",
                    "ScheduledTagsProperty": {
                        "$id": "30",
                        "$values": [
                            "13a17d90-f440-49bd-a84b-0c6ef45d1d04",
                            "d53ff52c-bfa4-4bb4-aaff-d09f95986cf9"
                        ],
                        "_type": "System.Collections.Generic.List\u00601[[System.Guid, System.Private.CoreLib]], System.Private.CoreLib"
                    },
                    "CompletedTagsProperty": {
                        "$id": "31",
                        "$values": [],
                        "_type": "System.Collections.Generic.List\u00601[[System.Guid, System.Private.CoreLib]], System.Private.CoreLib"
                    }
                },
                "activityState": {
                    "$id": "32",
                    "Items": {
                        "$id": "33",
                        "$values": [
                            {
                                "id": "1",
                                "name": "User 1"
                            },
                            {
                                "id": "2",
                                "name": "User 2"
                            }
                        ],
                        "_type": "System.Collections.Generic.List\u00601[[System.Object, System.Private.CoreLib]], System.Private.CoreLib"
                    }
                },
                "dynamicVariables": {
                    "$id": "34",
                    "$values": []
                },
                "status": "Running",
                "startedAt": "2024-04-24T17:50:20.4514158+00:00"
            },
            {
                "$id": "35",
                "id": "8145154905ebf214",
                "parentContextId": "9f159acbf8ed8cf9",
                "scheduledActivityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2",
                "ownerActivityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1",
                "properties": {
                    "$id": "36",
                    "CurrentIndex": 2,
                    "PersistentVariablesDictionary": {
                        "4d13475cb686436a812a5d6ec2fdcbb3": {
                            "$id": "37",
                            "id": "1",
                            "name": "User 1",
                            "_type": "Elsa.Demo.API.Workflows.User, API"
                        },
                        "4e5222962ece41f0b42acc96ae1df77e": 0,
                        "_type": "System.Collections.Generic.Dictionary\u00602[[System.String, System.Private.CoreLib],[System.Object, System.Private.CoreLib]], System.Private.CoreLib"
                    }
                },
                "activityState": {
                    "$id": "38"
                },
                "dynamicVariables": {
                    "$id": "39",
                    "$values": [
                        {
                            "$id": "40",
                            "id": "4d13475cb686436a812a5d6ec2fdcbb3",
                            "name": "CurrentValue",
                            "typeName": "Elsa.Demo.API.Workflows.User, API",
                            "storageDriverTypeName": "Elsa.Workflows.Services.WorkflowStorageDriver, Elsa.Workflows.Core"
                        },
                        {
                            "$id": "41",
                            "id": "4e5222962ece41f0b42acc96ae1df77e",
                            "name": "CurrentIndex",
                            "typeName": "Int32",
                            "value": "0",
                            "storageDriverTypeName": "Elsa.Workflows.Services.WorkflowStorageDriver, Elsa.Workflows.Core"
                        }
                    ]
                },
                "status": "Running",
                "startedAt": "2024-04-24T17:50:20.4559866+00:00"
            },
            {
                "$id": "42",
                "id": "50eefe4e0e36361e",
                "parentContextId": "9f159acbf8ed8cf9",
                "scheduledActivityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2",
                "ownerActivityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1",
                "properties": {
                    "$id": "43",
                    "CurrentIndex": 1,
                    "PersistentVariablesDictionary": {
                        "57ff27fb92cf4bfdb49f1631fc05bfcd": {
                            "$id": "44",
                            "id": "2",
                            "name": "User 2",
                            "_type": "Elsa.Demo.API.Workflows.User, API"
                        },
                        "91f4d35bba0a4ccba397dff4cbc3206e": 1,
                        "_type": "System.Collections.Generic.Dictionary\u00602[[System.String, System.Private.CoreLib],[System.Object, System.Private.CoreLib]], System.Private.CoreLib"
                    }
                },
                "activityState": {
                    "$id": "45"
                },
                "dynamicVariables": {
                    "$id": "46",
                    "$values": [
                        {
                            "$id": "47",
                            "id": "57ff27fb92cf4bfdb49f1631fc05bfcd",
                            "name": "CurrentValue",
                            "typeName": "Elsa.Demo.API.Workflows.User, API",
                            "storageDriverTypeName": "Elsa.Workflows.Services.WorkflowStorageDriver, Elsa.Workflows.Core"
                        },
                        {
                            "$id": "48",
                            "id": "91f4d35bba0a4ccba397dff4cbc3206e",
                            "name": "CurrentIndex",
                            "typeName": "Int32",
                            "value": "1",
                            "storageDriverTypeName": "Elsa.Workflows.Services.WorkflowStorageDriver, Elsa.Workflows.Core"
                        }
                    ]
                },
                "status": "Running",
                "startedAt": "2024-04-24T17:50:20.4573693+00:00",
                "tag": "d53ff52c-bfa4-4bb4-aaff-d09f95986cf9"
            },
            {
                "$id": "49",
                "id": "c56b45635a7d2ad0",
                "parentContextId": "50eefe4e0e36361e",
                "scheduledActivityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2:RunTask1",
                "ownerActivityNodeId": "Workflow1:Sequence1:ParallelForEach\u003CUser\u003E1:Sequence2",
                "properties": {
                    "$id": "50"
                },
                "activityState": {
                    "$id": "51",
                    "TaskName": "PromptVote",
                    "Payload": {
                        "User": {
                            "id": "2",
                            "name": "User 2"
                        }
                    }
                },
                "dynamicVariables": {
                    "$id": "52",
                    "$values": []
                },
                "status": "Running",
                "startedAt": "2024-04-24T17:50:20.4769724+00:00"
            }
        ]
    },
    "scheduledActivities": {
        "$id": "53",
        "$values": []
    },
    "executionLogSequence": 17,
    "input": {
        "$id": "54"
    },
    "output": {
        "$id": "55"
    },
    "properties": {
        "$id": "56"
    },
    "createdAt": "2024-04-24T17:50:20.3808319+00:00",
    "updatedAt": "2024-04-24T17:50:49.9720381+00:00",
    "finishedAt": "2024-04-24T17:50:49.9720381+00:00"
}

elsa-workflows / elsa-core