Long-running workflows don't save correctly in workflow_execution table

[jakub-borek]

Bug report

Long-running workflows are not saved correctly in workflow_execution table

Describe the bug

Long-running workflows after being executed get stuck in the invoking state in the workflow_execution table.

After running a hello-world long-running workflow from documentation, in console it shows as finished:

When I go to Admin UI -> Settings -> Workflows there is a new row stuck in Invoking state:

When I click on that workflow I see a screen with text "Loading...", probably because when I go to database -> workflow_execution table and refresh the table a few times the record appears and disappears, as if it was being deleted and added again.

System information

Medusa version (including plugins): clean install using 'npx create-medusa-app@preview' on 10.09.2024 Node.js version: 20.16.0 Database: PostgreSQL 16 Operating system: Windows 11 Browser (if relevant): Chrome

Steps to reproduce the behavior

1. Install latest Medusa 2.0 backend app
2. Copy the long running workflow example from documentation
3. Start the application and invoke route with workflow using Postman
4. Check Admin UI -> Settings -> Workflows. Workflow should show as "invoking". Reload a page few times - workflow appears and disappears.
5. Check workflow_execution table and refresh it a few times - record appears and disappears.

Expected behavior

Long running workflow after finishing should show as either done/success or failed state on Workflows Admin UI. Record in table should not be appearing and disappearing.

Context

Here's a copy of the row from workflow_execution table: workflow_execution.txt

[adrien2p]

We need to improve the doc on that topic.

but an async step that return a step response will auto complete itself instead of waiting for an external action to mark it as either success of failure. Also, since there is no retention time configured, the result of that workflow will be removed after completion.

Cc @shahednasser

[carlos-r-l-rodrigues]

We are still polishing the details to correctly display or not things in the admin, and there are many things to be documented yet.

But here is an updated version that you can play and better understand the behavior you are expecting.

import {
  createStep,
  createWorkflow,
  WorkflowResponse,
  StepResponse,
} from "@medusajs/workflows-sdk";
import { setTimeout } from 'node:timers/promises';

const step1 = createStep("step-1", async () => {
  return new StepResponse({});
});

const step2 = createStep(
  {
    name: "step-2",
    async: true,
  },
  async () => {
    await setTimeout(1000 * 15);
    return new StepResponse({});
  }
);

const step3 = createStep("step-3", async () => {
  return new StepResponse("Finished three steps");
});

const myWorkflow = createWorkflow({
  name: "hello-world",
  retentionTime: 99999
}, function () {
  step1();
  step2();
  const message = step3();

  return new WorkflowResponse({
    message,
  });
});

export default myWorkflow;

[adrien2p]

Carlos example will be more appropriate to test this (I am on my phone and cant really write examples 😅)

[bert-rstk]

We are still polishing the details to correctly display or not things in the admin, and there are many things to be documented yet.

But here is an updated version that you can play and better understand the behavior you are expecting.

Thank you for providing an example 👍 Do you have an example (or documentation) for how to set a retention time to persist the result of a workflow (as @adrien2p mentioned)?

[adrien2p]

We are still polishing the details to correctly display or not things in the admin, and there are many things to be documented yet.

But here is an updated version that you can play and better understand the behavior you are expecting.

Thank you for providing an example 👍 Do you have an example (or documentation) for how to set a retention time to persist the result of a workflow (as @adrien2p mentioned)?

It is part of the example above, it is an option of the workflow.

[bert-rstk]

It is part of the example above, it is an option of the workflow.

My bad, I missed the line that sets the value for retentionTime... thank you 🙏

[jakub-borek]

Thank you for quick response!

I tested this new code and now it works correctly, both with succeded and failed steps.

I'm closing this one as a resolved issue.