search

StackStorm / st2

StackStorm (aka "IFTTT for Ops") is event-driven automation for auto-remediation, incident responses, troubleshooting, deployments, and more for DevOps and SREs. Includes rules engine, workflow, 160 integration packs with 6000+ actions (see https://exchange.stackstorm.org) and ChatOps. Installer at https://docs.stackstorm.com/install/index.html
https://stackstorm.com/
Apache License 2.0
6.08k stars 747 forks source link

Orquesta workflow gets stuck in the running state #4968

Open igcherkaev opened 4 years ago

igcherkaev commented 4 years ago

SUMMARY

Sorta similar to #4954 but this time without empty arrays or anything. This time if subworkflow called from the parent workflow using "with-items" fails, the parent workflow won't finish if you intend it to finish with success, i.e. using "noop" transition.

STACKSTORM VERSION

$ st2 --version
st2 3.2.0, on Python 2.7.5
OS, environment, install method

CentOS 7 install following the docs at: https://docs.stackstorm.com/install/rhel7.html

Steps to reproduce the problem

Run the following workflow:

wf_orquesta_stuck2.meta.yaml:

---
pack: "playground"
name: "wf_orquesta_stuck2"
description: "Orquesta workflow gets stuck in running bug, st2 v3.2.0"
runner_type: orquesta
enabled: true
entry_point: "workflows/wf_orquesta_stuck2.yaml"
parameters:
  some_array:
    required: false
    default: ["a", "b"]
    type: "array"
    description: "Some array."

workflows/wf_orquesta_stuck2.yaml:

---
version: '1.0'

input:
  - some_array

output:
  - result: <% ctx().get('wf_result') %>
tasks:
  init_task:
    action: core.noop
    next:
      - when: <% succeeded() %>
        do:
          - subflow
  subflow:
    with:
      items: i in <% ctx().some_array %>
    action: playground.wf_orquesta_stuck2_sub
    next:
      - when: <% succeeded() %>
        publish:
          - wf_result: 'succeeded'
        do:
          - finish

      - when: <% failed() %>
        publish:
          - wf_result: 'failed'
        do:
          - finish_failed

  finish:
    action: core.noop

  finish_failed:
    action: core.noop

And the subworkflow used is

__wf_orquesta_stuck2_sub.meta.yaml__:

---
pack: "playground"
name: "wf_orquesta_stuck2_sub"
description: "Orquesta sub workflow that gets main workflow stuck in running, st2 v3.2.0"
runner_type: orquesta
enabled: true
entry_point: "workflows/wf_orquesta_stuck2_sub.yaml"

__workflows/wf_orquesta_stuck2_sub.yaml__:

---
version: '1.0'

tasks:
  subtask1:
    action: core.local
    input:
      cmd: 'exit 1'
    next:
      - when: <% failed() %>
        do:
          - subtask_failed
  subtask_failed:
    action: core.local
    input:
      cmd: 'exit 1'
    next:
      - when: <% succeeded() %>
        do:
          - good

  good:
    action: core.noop

Expected Results

The parent (main) workflow to finish with success.

Actual Results

The parent (main) workflow gets stuck in the running state and never finishes.

st2workflowengine.log:

2020-06-15 10:53:27,997 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Processing request for workflow execution.
2020-06-15 10:53:28,014 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Requesting conductor to start running workflow execution.
2020-06-15 10:53:28,015 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Identifying next set (0) of tasks for workflow execution in status "running".
2020-06-15 10:53:28,018 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Identified the following set of tasks to execute next: init_task (route 0)
2020-06-15 10:53:28,018 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Mark task "init_task", route "0", in conductor as running.
2020-06-15 10:53:28,018 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Updating workflow execution from status "requested" to "running".
2020-06-15 10:53:28,058 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Requesting execution for task "init_task", route "0".
2020-06-15 10:53:28,059 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Processing task execution request for task "init_task", route "0".
2020-06-15 10:53:28,066 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Task execution "5ee7b59897a7163e06f9c5de" created for task "init_task", route "0".
2020-06-15 10:53:28,183 140025611212112 AUDIT action [-] Action execution requested. LiveAction.id=5ee7b59897a7163e06f9c5df, ActionExecution.id=5ee7b59897a7163e06f9c5e0
2020-06-15 10:53:28,183 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e0" requested for task "init_task", route "0".
2020-06-15 10:53:28,200 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Identifying next set (1) of tasks for workflow execution in status "running".
2020-06-15 10:53:28,201 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] No tasks identified to execute next.
2020-06-15 10:53:28,269 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e0" for task "init_task" is updated and in "scheduled" state.
2020-06-15 10:53:28,347 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e0" for task "init_task" is updated and in "running" state.
2020-06-15 10:53:28,421 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e0" for task "init_task" is updated and in "succeeded" state.
2020-06-15 10:53:28,435 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Handling completion of action execution "5ee7b59897a7163e06f9c5e0" in status "succeeded" for task "init_task", route "0".
2020-06-15 10:53:28,475 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Publish task "init_task", route "0", with status "succeeded" to conductor.
2020-06-15 10:53:28,511 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Identifying next set (0) of tasks after completion of task "init_task", route "0".
2020-06-15 10:53:28,515 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Identified the following set of tasks to execute next: subflow (route 0)
2020-06-15 10:53:28,516 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Mark task "subflow", route "0", in conductor as running.
2020-06-15 10:53:28,516 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Mark task "subflow", route "0", item "0" in conductor as running.
2020-06-15 10:53:28,516 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Mark task "subflow", route "0", item "1" in conductor as running.
2020-06-15 10:53:28,555 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Requesting execution for task "subflow", route "0".
2020-06-15 10:53:28,555 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Processing task execution request for task "subflow", route "0".
2020-06-15 10:53:28,563 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Task execution "5ee7b59897a7163e06f9c5e1" created for task "subflow", route "0".
2020-06-15 10:53:28,676 140025611212112 AUDIT action [-] Action execution requested. LiveAction.id=5ee7b59897a7163e06f9c5e2, ActionExecution.id=5ee7b59897a7163e06f9c5e3
2020-06-15 10:53:28,676 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e3" requested for task "subflow", route "0".
2020-06-15 10:53:28,807 140025611211472 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e3" for task "subflow" is updated and in "scheduled" state.
2020-06-15 10:53:28,814 140025611212112 AUDIT action [-] Action execution requested. LiveAction.id=5ee7b59897a7163e06f9c5e4, ActionExecution.id=5ee7b59897a7163e06f9c5e5
2020-06-15 10:53:28,814 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e5" requested for task "subflow", route "0".
2020-06-15 10:53:28,834 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] Identifying next set (1) of tasks for workflow execution in status "running".
2020-06-15 10:53:28,839 140025611212112 INFO workflows [-] [5ee7b597b8572463cfc70444] No tasks identified to execute next.
2020-06-15 10:53:28,872 140025611212272 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e3" for task "subflow" is updated and in "running" state.
2020-06-15 10:53:28,893 140025611212272 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e5" for task "subflow" is updated and in "scheduled" state.
2020-06-15 10:53:28,980 140025611212272 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e5" for task "subflow" is updated and in "running" state.
2020-06-15 10:53:28,983 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Processing request for workflow execution.
2020-06-15 10:53:28,997 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Requesting conductor to start running workflow execution.
2020-06-15 10:53:28,997 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Identifying next set (0) of tasks for workflow execution in status "running".
2020-06-15 10:53:29,000 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Identified the following set of tasks to execute next: subtask1 (route 0)
2020-06-15 10:53:29,000 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Mark task "subtask1", route "0", in conductor as running.
2020-06-15 10:53:29,000 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Updating workflow execution from status "requested" to "running".
2020-06-15 10:53:29,046 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Requesting execution for task "subtask1", route "0".
2020-06-15 10:53:29,047 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Processing task execution request for task "subtask1", route "0".
2020-06-15 10:53:29,055 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Task execution "5ee7b59997a7163e06f9c5e6" created for task "subtask1", route "0".
2020-06-15 10:53:29,089 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Processing request for workflow execution.
2020-06-15 10:53:29,102 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Requesting conductor to start running workflow execution.
2020-06-15 10:53:29,103 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Identifying next set (0) of tasks for workflow execution in status "running".
2020-06-15 10:53:29,105 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Identified the following set of tasks to execute next: subtask1 (route 0)
2020-06-15 10:53:29,105 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Mark task "subtask1", route "0", in conductor as running.
2020-06-15 10:53:29,106 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Updating workflow execution from status "requested" to "running".
2020-06-15 10:53:29,201 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Requesting execution for task "subtask1", route "0".
2020-06-15 10:53:29,202 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Processing task execution request for task "subtask1", route "0".
2020-06-15 10:53:29,262 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Task execution "5ee7b59997a7163e06f9c5e9" created for task "subtask1", route "0".
2020-06-15 10:53:29,290 140025611212752 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e3" for task "subflow" is updated and in "running" state.
2020-06-15 10:53:29,299 140025611212272 AUDIT action [-] Action execution requested. LiveAction.id=5ee7b59997a7163e06f9c5e7, ActionExecution.id=5ee7b59997a7163e06f9c5e8
2020-06-15 10:53:29,300 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Action execution "5ee7b59997a7163e06f9c5e8" requested for task "subtask1", route "0".
2020-06-15 10:53:29,329 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Identifying next set (1) of tasks for workflow execution in status "running".
2020-06-15 10:53:29,330 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] No tasks identified to execute next.
2020-06-15 10:53:29,409 140025611212752 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Action execution "5ee7b59997a7163e06f9c5e8" for task "subtask1" is updated and in "scheduled" state.
2020-06-15 10:53:29,458 140025611214032 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e5" for task "subflow" is updated and in "running" state.
2020-06-15 10:53:29,469 140025611212592 AUDIT action [-] Action execution requested. LiveAction.id=5ee7b59997a7163e06f9c5ea, ActionExecution.id=5ee7b59997a7163e06f9c5eb
2020-06-15 10:53:29,469 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Action execution "5ee7b59997a7163e06f9c5eb" requested for task "subtask1", route "0".
2020-06-15 10:53:29,487 140025611212912 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Action execution "5ee7b59997a7163e06f9c5e8" for task "subtask1" is updated and in "running" state.
2020-06-15 10:53:29,498 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Identifying next set (1) of tasks for workflow execution in status "running".
2020-06-15 10:53:29,498 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] No tasks identified to execute next.
2020-06-15 10:53:29,615 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Action execution "5ee7b59997a7163e06f9c5eb" for task "subtask1" is updated and in "scheduled" state.
2020-06-15 10:53:29,683 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Action execution "5ee7b59997a7163e06f9c5eb" for task "subtask1" is updated and in "running" state.
2020-06-15 10:53:29,993 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Action execution "5ee7b59997a7163e06f9c5e8" for task "subtask1" is updated and in "failed" state.
2020-06-15 10:53:30,010 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Action execution "5ee7b59997a7163e06f9c5eb" for task "subtask1" is updated and in "failed" state.
2020-06-15 10:53:30,021 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Handling completion of action execution "5ee7b59997a7163e06f9c5e8" in status "failed" for task "subtask1", route "0".
2020-06-15 10:53:30,044 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Handling completion of action execution "5ee7b59997a7163e06f9c5eb" in status "failed" for task "subtask1", route "0".
2020-06-15 10:53:30,080 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Publish task "subtask1", route "0", with status "failed" to conductor.
2020-06-15 10:53:30,126 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Identifying next set (0) of tasks after completion of task "subtask1", route "0".
2020-06-15 10:53:30,129 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Identified the following set of tasks to execute next: subtask_failed (route 0)
2020-06-15 10:53:30,129 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Mark task "subtask_failed", route "0", in conductor as running.
2020-06-15 10:53:30,151 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Publish task "subtask1", route "0", with status "failed" to conductor.
2020-06-15 10:53:30,197 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Identifying next set (0) of tasks after completion of task "subtask1", route "0".
2020-06-15 10:53:30,200 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Identified the following set of tasks to execute next: subtask_failed (route 0)
2020-06-15 10:53:30,200 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Mark task "subtask_failed", route "0", in conductor as running.
2020-06-15 10:53:30,236 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Requesting execution for task "subtask_failed", route "0".
2020-06-15 10:53:30,236 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Processing task execution request for task "subtask_failed", route "0".
2020-06-15 10:53:30,246 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Task execution "5ee7b59a97a7163e06f9c5ec" created for task "subtask_failed", route "0".
2020-06-15 10:53:30,275 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Requesting execution for task "subtask_failed", route "0".
2020-06-15 10:53:30,275 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Processing task execution request for task "subtask_failed", route "0".
2020-06-15 10:53:30,292 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Task execution "5ee7b59a97a7163e06f9c5ed" created for task "subtask_failed", route "0".
2020-06-15 10:53:30,458 140025611214512 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e3" for task "subflow" is updated and in "running" state.
2020-06-15 10:53:30,459 140025611212592 AUDIT action [-] Action execution requested. LiveAction.id=5ee7b59a97a7163e06f9c5ee, ActionExecution.id=5ee7b59a97a7163e06f9c5ef
2020-06-15 10:53:30,460 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Action execution "5ee7b59a97a7163e06f9c5ef" requested for task "subtask_failed", route "0".
2020-06-15 10:53:30,481 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Identifying next set (1) of tasks for workflow execution in status "running".
2020-06-15 10:53:30,482 140025611212592 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] No tasks identified to execute next.
2020-06-15 10:53:30,547 140025611211152 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e5" for task "subflow" is updated and in "running" state.
2020-06-15 10:53:30,556 140025611214352 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Action execution "5ee7b59a97a7163e06f9c5ef" for task "subtask_failed" is updated and in "scheduled" state.
2020-06-15 10:53:30,559 140025611212272 AUDIT action [-] Action execution requested. LiveAction.id=5ee7b59a97a7163e06f9c5f0, ActionExecution.id=5ee7b59a97a7163e06f9c5f1
2020-06-15 10:53:30,559 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Action execution "5ee7b59a97a7163e06f9c5f1" requested for task "subtask_failed", route "0".
2020-06-15 10:53:30,576 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Identifying next set (1) of tasks for workflow execution in status "running".
2020-06-15 10:53:30,576 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] No tasks identified to execute next.
2020-06-15 10:53:30,629 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Action execution "5ee7b59a97a7163e06f9c5ef" for task "subtask_failed" is updated and in "running" state.
2020-06-15 10:53:30,655 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Action execution "5ee7b59a97a7163e06f9c5f1" for task "subtask_failed" is updated and in "scheduled" state.
2020-06-15 10:53:30,746 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Action execution "5ee7b59a97a7163e06f9c5f1" for task "subtask_failed" is updated and in "running" state.
2020-06-15 10:53:30,780 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Action execution "5ee7b59a97a7163e06f9c5ef" for task "subtask_failed" is updated and in "failed" state.
2020-06-15 10:53:30,798 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Handling completion of action execution "5ee7b59a97a7163e06f9c5ef" in status "failed" for task "subtask_failed", route "0".
2020-06-15 10:53:30,858 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Publish task "subtask_failed", route "0", with status "failed" to conductor.
2020-06-15 10:53:30,860 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Updating workflow execution from status "running" to "failed".
2020-06-15 10:53:30,893 140025611212272 ERROR workflows [-] [5ee7b59897a7163e06f9c5e3] Workflow execution completed with errors. (task_id=u'subtask1',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:30,894 140025611212272 ERROR workflows [-] [5ee7b59897a7163e06f9c5e3] Workflow execution completed with errors. (task_id=u'subtask_failed',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:30,897 140025611853648 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Action execution "5ee7b59a97a7163e06f9c5f1" for task "subtask_failed" is updated and in "failed" state.
2020-06-15 10:53:30,917 140025611853648 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Handling completion of action execution "5ee7b59a97a7163e06f9c5f1" in status "failed" for task "subtask_failed", route "0".
2020-06-15 10:53:30,962 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Workflow action execution is completed and invoking post run.
2020-06-15 10:53:30,962 140025611212272 INFO utils [-] Invoking post run for action execution 5ee7b59897a7163e06f9c5e2.
2020-06-15 10:53:30,966 140025611212272 INFO utils [-] Action execution 5ee7b59897a7163e06f9c5e2 runs wf_orquesta_stuck2_sub of runner type orquesta.
2020-06-15 10:53:30,997 140025611212912 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e3" for task "subflow" is updated and in "failed" state.
2020-06-15 10:53:31,035 140025611853648 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Publish task "subtask_failed", route "0", with status "failed" to conductor.
2020-06-15 10:53:31,038 140025611853648 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Updating workflow execution from status "running" to "failed".
2020-06-15 10:53:31,058 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] Identifying next set (0) of tasks after completion of task "subtask_failed", route "0".
2020-06-15 10:53:31,059 140025611212272 INFO workflows [-] [5ee7b59897a7163e06f9c5e3] No tasks identified to execute next.
2020-06-15 10:53:31,169 140025611212912 INFO workflows [-] [5ee7b597b8572463cfc70444] Handling completion of action execution "5ee7b59897a7163e06f9c5e3" in status "failed" for task "subflow", route "0".
2020-06-15 10:53:31,192 140025611212272 ERROR workflows [-] [5ee7b59897a7163e06f9c5e3] Workflow execution completed with errors. (task_id=u'subtask1',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,193 140025611212272 ERROR workflows [-] [5ee7b59897a7163e06f9c5e3] Workflow execution completed with errors. (task_id=u'subtask_failed',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,195 140025611853648 ERROR workflows [-] [5ee7b59897a7163e06f9c5e5] Workflow execution completed with errors. (task_id=u'subtask1',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,196 140025611853648 ERROR workflows [-] [5ee7b59897a7163e06f9c5e5] Workflow execution completed with errors. (task_id=u'subtask_failed',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,241 140025611212912 INFO workflows [-] [5ee7b597b8572463cfc70444] Publish task "subflow", route "0", with status "running" to conductor.
2020-06-15 10:53:31,301 140025611212912 INFO workflows [-] [5ee7b597b8572463cfc70444] Identifying next set (0) of tasks after completion of task "subflow", route "0".
2020-06-15 10:53:31,307 140025611212912 INFO workflows [-] [5ee7b597b8572463cfc70444] No tasks identified to execute next.
2020-06-15 10:53:31,340 140025611853648 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Workflow action execution is completed and invoking post run.
2020-06-15 10:53:31,340 140025611853648 INFO utils [-] Invoking post run for action execution 5ee7b59897a7163e06f9c5e4.
2020-06-15 10:53:31,351 140025611853648 INFO utils [-] Action execution 5ee7b59897a7163e06f9c5e4 runs wf_orquesta_stuck2_sub of runner type orquesta.
2020-06-15 10:53:31,389 140025611212272 ERROR workflows [-] [5ee7b59897a7163e06f9c5e3] Workflow execution completed with errors. (task_id=u'subtask1',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,389 140025611212272 ERROR workflows [-] [5ee7b59897a7163e06f9c5e3] Workflow execution completed with errors. (task_id=u'subtask_failed',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,402 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59897a7163e06f9c5e5" for task "subflow" is updated and in "failed" state.
2020-06-15 10:53:31,413 140025611853648 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] Identifying next set (0) of tasks after completion of task "subtask_failed", route "0".
2020-06-15 10:53:31,413 140025611853648 INFO workflows [-] [5ee7b59897a7163e06f9c5e5] No tasks identified to execute next.
2020-06-15 10:53:31,459 140025611853648 ERROR workflows [-] [5ee7b59897a7163e06f9c5e5] Workflow execution completed with errors. (task_id=u'subtask1',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,460 140025611853648 ERROR workflows [-] [5ee7b59897a7163e06f9c5e5] Workflow execution completed with errors. (task_id=u'subtask_failed',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,545 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Handling completion of action execution "5ee7b59897a7163e06f9c5e5" in status "failed" for task "subflow", route "0".
2020-06-15 10:53:31,547 140025611853648 ERROR workflows [-] [5ee7b59897a7163e06f9c5e5] Workflow execution completed with errors. (task_id=u'subtask1',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,548 140025611853648 ERROR workflows [-] [5ee7b59897a7163e06f9c5e5] Workflow execution completed with errors. (task_id=u'subtask_failed',type=u'error',result={u'failed': True, u'stderr': u'', u'return_code': 1, u'succeeded': False, u'stdout': u''},message=u'Execution failed. See result for details.')
2020-06-15 10:53:31,618 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Publish task "subflow", route "0", with status "failed" to conductor.
2020-06-15 10:53:31,665 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Identifying next set (0) of tasks after completion of task "subflow", route "0".
2020-06-15 10:53:31,671 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Identified the following set of tasks to execute next: finish_failed (route 0)
2020-06-15 10:53:31,671 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Mark task "finish_failed", route "0", in conductor as running.
2020-06-15 10:53:31,715 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Requesting execution for task "finish_failed", route "0".
2020-06-15 10:53:31,715 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Processing task execution request for task "finish_failed", route "0".
2020-06-15 10:53:31,726 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Task execution "5ee7b59b97a7163e06f9c5f2" created for task "finish_failed", route "0".
2020-06-15 10:53:31,845 140025611214352 AUDIT action [-] Action execution requested. LiveAction.id=5ee7b59b97a7163e06f9c5f3, ActionExecution.id=5ee7b59b97a7163e06f9c5f4
2020-06-15 10:53:31,846 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59b97a7163e06f9c5f4" requested for task "finish_failed", route "0".
2020-06-15 10:53:31,867 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Identifying next set (1) of tasks for workflow execution in status "running".
2020-06-15 10:53:31,872 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] No tasks identified to execute next.
2020-06-15 10:53:31,907 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59b97a7163e06f9c5f4" for task "finish_failed" is updated and in "scheduled" state.
2020-06-15 10:53:31,966 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59b97a7163e06f9c5f4" for task "finish_failed" is updated and in "running" state.
2020-06-15 10:53:32,044 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Action execution "5ee7b59b97a7163e06f9c5f4" for task "finish_failed" is updated and in "succeeded" state.
2020-06-15 10:53:32,061 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Handling completion of action execution "5ee7b59b97a7163e06f9c5f4" in status "succeeded" for task "finish_failed", route "0".
2020-06-15 10:53:32,115 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Publish task "finish_failed", route "0", with status "succeeded" to conductor.
2020-06-15 10:53:32,154 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] Identifying next set (0) of tasks after completion of task "finish_failed", route "0".
2020-06-15 10:53:32,159 140025611214352 INFO workflows [-] [5ee7b597b8572463cfc70444] No tasks identified to execute next.
guzzijones commented 4 years ago

There is some documentation on this exact use case. Try changing finish_failed to continue