Workflow not going forward after DO_WHILE execution completion

[appunni-old]

Describe the bug

Details Conductor version: 3.15.0 Persistence implementation: MySQL Queue implementation: MySql Lock: Redis Workflow definition:

{
  "createTime": 1701669713675,
  "updateTime": 1701669746737,
  "createdBy": "owner@email.com",
  "updatedBy": "owner@email.com",
  "accessPolicy": {},
  "name": "test_do_while",
  "description": "Workflow details",
  "version": 2,
  "tasks": [
    {
      "name": "default_do_while",
      "taskReferenceName": "task_1__loop_databricks",
      "inputParameters": {},
      "type": "DO_WHILE",
      "startDelay": 0,
      "optional": false,
      "asyncComplete": false,
      "loopCondition": "if ($.task_1__loop_databricks['iteration'] < 1) { true; } else { false; }",
      "loopOver": [
        {
          "name": "default_sleep",
          "taskReferenceName": "task_1__wait_databricks",
          "inputParameters": {
            "duration": "20 seconds",
            "tenantId": "csit"
          },
          "type": "WAIT",
          "startDelay": 0,
          "optional": false,
          "asyncComplete": false
        }
      ]
    },
    {
      "name": "default_sleep",
      "taskReferenceName": "task_2__wait_databricks",
      "inputParameters": {
        "duration": "20 seconds",
        "tenantId": "csit"
      },
      "type": "WAIT",
      "startDelay": 0,
      "optional": false,
      "asyncComplete": false
    }
  ],
  "inputParameters": [],
  "outputParameters": {},
  "schemaVersion": 2,
  "restartable": true,
  "workflowStatusListenerEnabled": false,
  "ownerEmail": "owner@email.com",
  "timeoutPolicy": "ALERT_ONLY",
  "timeoutSeconds": 0,
  "variables": {},
  "inputTemplate": {}
}

Task definition: System Event handler definition:

To Reproduce Steps to reproduce the behavior:

-> Execute the above workflow -> Wait for 20 seconds for the WAIT task completion

Expected behavior WAIT task outside the DO_WHILE was supposed to execute

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Testing was sone after merging #3878

[appunni-old]

This may be related to #3876 . Again Double underscore may be causing issue

[appunni-old]

{
  "createTime": 1701670338519,
  "updateTime": 1701669746737,
  "createdBy": "owner@email.com",
  "updatedBy": "owner@email.com",
  "accessPolicy": {},
  "name": "test_do_while_2",
  "description": "Workflow details",
  "version": 1,
  "tasks": [
    {
      "name": "default_do_while",
      "taskReferenceName": "task_1_loop_databricks",
      "inputParameters": {},
      "type": "DO_WHILE",
      "startDelay": 0,
      "optional": false,
      "asyncComplete": false,
      "loopCondition": "if ($.task_1_loop_databricks['iteration'] < 1) { true; } else { false; }",
      "loopOver": [
        {
          "name": "default_sleep",
          "taskReferenceName": "task_1_wait_databricks",
          "inputParameters": {
            "duration": "20 seconds",
            "tenantId": "csit"
          },
          "type": "WAIT",
          "startDelay": 0,
          "optional": false,
          "asyncComplete": false
        }
      ]
    },
    {
      "name": "default_sleep",
      "taskReferenceName": "task_2_wait_databricks",
      "inputParameters": {
        "duration": "20 seconds",
        "tenantId": "csit"
      },
      "type": "WAIT",
      "startDelay": 0,
      "optional": false,
      "asyncComplete": false
    }
  ],
  "inputParameters": [],
  "outputParameters": {},
  "schemaVersion": 2,
  "restartable": true,
  "workflowStatusListenerEnabled": false,
  "ownerEmail": "owner@email.com",
  "timeoutPolicy": "ALERT_ONLY",
  "timeoutSeconds": 0,
  "variables": {},
  "inputTemplate": {}
}

using this worked.

[appunni-old]

            List<TaskModel> loopOverTaskList =
                    workflow.getTasks().stream()
                            .filter(
                                    t ->
                                            TaskUtils.removeIterationFromTaskRefName(
                                                            t.getReferenceTaskName())
                                                    .equals(taskRefName))
                            .collect(Collectors.toList());

This could be the bug here, as they are using removeIterationFromTaskRefName for checking if it's an iterable task. But the method itself is too fragile implementation.

SimpleActionProcessor:132

[gajendrangnanasekaran]

HI can you help me to do setup conductor with postgres

[appunni-old]

Hi @gajendrangnanasekaran you can use docker-compose to set it up . Clone this repo and run

docker-compose -f docker/docker-compose-postgres.yaml up -d

To kill run

docker-compose -f docker/docker-compose-postgres.yaml kill

If you want to run local setup for development, remove conductor:server from docker-compose file and create a new docker-compose file and run that docker-compose file using the similar command.

Then start up local server by running below command CONDUCTOR_CONFIG_FILE=docker/server/config/config-postgres-modified.properties ./gradlew bootRun

config-postgres-modified.properties must be pointing to localhost:6432 and localhost:9201 as postgres and elastic search

if you want to run the local server on intellij here is how it looks like The second line below jdk selection is the VM options

for mysql support I have added // community runtimeOnly("com.netflix.conductor:conductor-mysql-persistence:${revConductor}")

in the server/build.gradle dependencies. That's all