danielgerlag / workflow-core

Lightweight workflow engine for .NET Standard
MIT License
5.39k stars 1.2k forks source link

Workflow is too large to save #1082

Open wizd opened 2 years ago

wizd commented 2 years ago

Describe the bug When workflow size is too large, its unable to save it. Mongodb returns "Resulting document after update is larger than 16777216".

To Reproduce To create a document with size > 16777216.

Expected behavior There should be a chance, call back or event handler to let program to deal with the situation.

Additional context

C# exception log:

WorkflowCore.Services.BackgroundTasks.WorkflowConsumer[0] Error executing item 620a27c81070df48926dccaa - A write operation resulted in an error. WriteError: { Category : "Uncategorized", Code : 17419, Message : "Resulting document after update is larger than 16777216" }. MongoDB.Driver.MongoWriteException: A write operation resulted in an error. WriteError: { Category : "Uncategorized", Code : 17419, Message : "Resulting document after update is larger than 16777216" }. ---> MongoDB.Driver.MongoBulkWriteException1[WorkflowCore.Models.WorkflowInstance]: A bulk write operation resulted in one or more errors. WriteErrors: [ { Category : "Uncategorized", Code : 17419, Message : "Resulting document after update is larger than 16777216" } ]. at MongoDB.Driver.MongoCollectionImpl1.BulkWriteAsync(IClientSessionHandle session, IEnumerable1 requests, BulkWriteOptions options, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.UsingImplicitSessionAsync[TResult](Func2 funcAsync, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionBase1.ReplaceOneAsync(FilterDefinition1 filter, TDocument replacement, ReplaceOptions options, Func3 bulkWriteAsync) --- End of inner exception stack trace --- at MongoDB.Driver.MongoCollectionBase1.ReplaceOneAsync(FilterDefinition1 filter, TDocument replacement, ReplaceOptions options, Func3 bulkWriteAsync) at WorkflowCore.Persistence.MongoDB.Services.MongoPersistenceProvider.PersistWorkflow(WorkflowInstance workflow, CancellationToken cancellationToken) at WorkflowCore.Services.BackgroundTasks.WorkflowConsumer.ProcessItem(String itemId, CancellationToken cancellationToken) at WorkflowCore.Services.BackgroundTasks.WorkflowConsumer.ProcessItem(String itemId, CancellationToken cancellationToken) at WorkflowCore.Services.BackgroundTasks.QueueConsumer.ExecuteItem(String itemId, EventWaitHandle waitHandle, Activity activity)`

Anthony-Michel commented 2 years ago

In case that helps, seems you just ran into the maximum limit of a BSON doc in json, which is more a db concern or a model concern, how change your db behavior, or split your json in multiple parts https://www.mongodb.com/docs/manual/reference/limits/#mongodb-limit-BSON-Document-Size