MongoDB: Bulk Write Operation Error with '$type' Field in Elsa Workflows

[dupies]

Description: I encountered an issue when trying to run a workflow using MongoDB for persistence in my Elsa project. The error does not occur when using SQLite. Below are the details of my setup and the error message.

Error Message: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1] An unhandled exception has occurred while executing the request. MongoDB.Driver.MongoBulkWriteException1[Elsa.Workflows.Runtime.Entities.StoredTrigger]: A bulk write operation resulted in one or more errors. WriteErrors: [ { Category : "Uncategorized", Code : 52, Message : "The dollar ($) prefixed field '$type' in 'Payload.$type' is not valid for storage." } ]. at MongoDB.Driver.MongoCollectionImpl1.BulkWriteAsync(IClientSessionHandle session, IEnumerable`1 requests, BulkWriteOptions options, CancellationToken cancellationToken)

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <RuntimeFrameworkVersion>8.0.0</RuntimeFrameworkVersion>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <GenerateDocumentationFile>True</GenerateDocumentationFile>
    <DocumentationFile>C:\nugget rewrite project\JustAtestApp\Docs\ORVWorkflowAdminServer.xml</DocumentationFile>
    <IsPackable>true</IsPackable>
    <StaticWebAssetsEnabled>false</StaticWebAssetsEnabled>
  </PropertyGroup>
  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <Optimize>True</Optimize>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
    <Optimize>False</Optimize>
  </PropertyGroup>

  <ItemGroup>
    <Content Remove="Properties\launchSettings.json" />
    <Content Remove="Properties\launchSettings_old.json" />
  </ItemGroup>

  <ItemGroup>
    <_WebToolingArtifacts Remove="Properties\launchSettings.json" />
    <_WebToolingArtifacts Remove="Properties\launchSettings_old.json" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Elsa" Version="3.1.3" />
    <PackageReference Include="Elsa.Common" Version="3.1.3" />
    <PackageReference Include="Elsa.CSharp" Version="3.1.3" />
    <PackageReference Include="Elsa.Expressions" Version="3.1.3" />
    <PackageReference Include="Elsa.Features" Version="3.1.3" />
    <PackageReference Include="Elsa.Http" Version="3.1.3" />
    <PackageReference Include="Elsa.Identity" Version="3.1.3" />
    <PackageReference Include="Elsa.JavaScript" Version="3.1.3" />
    <PackageReference Include="Elsa.Labels" Version="3.1.3" />
    <PackageReference Include="Elsa.Liquid" Version="3.1.3" />
    <PackageReference Include="Elsa.MongoDb" Version="3.1.3" />
    <PackageReference Include="Elsa.SasTokens" Version="3.1.3" />
    <PackageReference Include="Elsa.Scheduling" Version="3.1.3" />
    <PackageReference Include="Elsa.Workflows.Api" Version="3.1.3" />
    <PackageReference Include="Elsa.Workflows.Core" Version="3.1.3" />
    <PackageReference Include="Elsa.Workflows.Management" Version="3.1.3" />
    <PackageReference Include="Elsa.Workflows.Runtime" Version="3.1.3" />
    <PackageReference Include="FluentStorage" Version="5.5.1" />
    <PackageReference Include="Fluid.Core" Version="2.11.1" />
    <PackageReference Include="Jint" Version="4.0.1" />
    <PackageReference Include="Microsoft.Extensions.Options" Version="8.0.2" />
    <PackageReference Include="MongoDB.Driver" Version="2.28.0" />
    <PackageReference Include="MongoDB.Driver.Core.Extensions.DiagnosticSources" Version="1.5.0" />
    <PackageReference Include="MongoDB.Driver.Extensions" Version="2.0.2" />
    <PackageReference Include="Open.Linq.AsyncExtensions" Version="1.2.0" />
    <PackageReference Include="Risk.Common.Classes" Version="1.0.102" />
    <PackageReference Include="Risk.NetCore.ServiceUtilities" Version="8.0.0-RC-2" />
  </ItemGroup>

</Project>

Program.cs:

using Elsa.Extensions;
using Elsa.MongoDb.Extensions;
using Elsa.MongoDb.Modules.Management;
using Elsa.MongoDb.Modules.Runtime;

var builder = WebApplication.CreateBuilder(args);

builder.Configuration
    .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
    .AddJsonFile($"appsettings.{builder.Environment.EnvironmentName}.json", optional: true, reloadOnChange: true)
    .AddEnvironmentVariables();

builder.Services.AddElsa(elsa =>
{
    var configuration = builder.Configuration;
    elsa.UseMongoDb(configuration.GetConnectionString("MongoDb"));

    // Configure Management layer to use MongoDB.
    elsa.UseWorkflowManagement(management => management.UseMongoDb());

    // Configure Runtime layer to use MongoDB.
    elsa.UseWorkflowRuntime(runtime => runtime.UseMongoDb());

    // Default Identity features for authentication/authorization.
    elsa.UseIdentity(identity =>
    {
        identity.TokenOptions = options => options.SigningKey = "sufficiently-large-secret-signing-key"; // This key needs to be at least 256 bits long.
        identity.UseAdminUserProvider();
    });

    // Configure ASP.NET authentication/authorization.
    elsa.UseDefaultAuthentication(auth => auth.UseAdminApiKey());

    // Expose Elsa API endpoints.
    elsa.UseWorkflowsApi();

    // Setup a SignalR hub for real-time updates from the server.
    elsa.UseRealTimeWorkflows();

    // Enable C# workflow expressions
    elsa.UseCSharp();

    // Enable HTTP activities.
    elsa.UseHttp();

    // Use timer activities.
    elsa.UseScheduling();

    // Register custom activities from the application, if any.
    elsa.AddActivitiesFrom<Program>();

    // Register custom workflows from the application, if any.
    elsa.AddWorkflowsFrom<Program>();
});

// Configure CORS to allow designer app hosted on a different origin to invoke the APIs.
builder.Services.AddCors(cors => cors
    .AddDefaultPolicy(policy => policy
        .AllowAnyOrigin() // For demo purposes only. Use a specific origin instead.
        .AllowAnyHeader()
        .AllowAnyMethod()
        .WithExposedHeaders("*"))); // Required for Elsa Studio in order to support running workflows from the designer. Alternatively, you can use the `*` wildcard to expose all headers or `x-elsa-workflow-instance-id`.

// Add Health Checks.
builder.Services.AddHealthChecks();

// Build the web application.
var app = builder.Build();

// Configure web application's middleware pipeline.
app.UseCors();
app.UseRouting(); // Required for SignalR.
app.UseAuthentication();
app.UseAuthorization();
app.UseWorkflowsApi(); // Use Elsa API endpoints.
app.UseWorkflows(); // Use Elsa middleware to handle HTTP requests mapped to HTTP Endpoint activities.
app.UseWorkflowsSignalRHubs(); // Optional SignalR integration. Elsa Studio uses SignalR to receive real-time updates from the server.

// Map the health check endpoint.
app.MapHealthChecks("/health");

app.Run();

Steps to Reproduce:

1. Set up a new Elsa Server project with the provided ElsaServer.csproj and Program.cs files.
2. Configure MongoDB connection string in appsettings.json.
3. Run the application.
4. Create and run a new Elsa Studio Project
5. Create a new Workflow and try run the workflow

Additional Information: • The error does not occur when using SQLite for persistence. • The MongoDB server is running locally at mongodb://localhost:27017/workflow.

Expected Behavior: The workflow should execute without any errors.

Actual Behavior: A MongoBulkWriteException is thrown with the message: "The dollar ($) prefixed field '$type' in 'Payload.$type' is not valid for storage." Environment: • Elsa Version: 3.1.3 • Elsa.MongoDb 3.1.3 • .NET Version: 8.0.0

Elsa Server:

Elsa Studio

[dupies]

The problem was that my MongoDB database version was 3.7, which does not support top-level fields with a dollar sign ($). MongoDB versions 3.0 and 4.0 have similar restrictions. To resolve this issue, I had to upgrade my MongoDB to version 5.0, which supports such fields.