Hang on "Workflow processing has been postponed due to max parallelism limit"

[Guillermogsjc]

Pre-requisites

• [X] I have double-checked my configuration
• [X] I can confirm the issues exists when I tested with :latest
• [ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

onExit pod hangs for 20 hours with Workflow processing has been postponed due to max parallelism limit, despite being 2 workflows active with few pods, and having 1000 as namespaceParallelism in the workflow-controller configmap

Version

v3.4.11

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

-

Logs from the workflow controller

over 20 hours of 
"Workflow processing has been postponed due to max parallelism limit"

Logs from in your workflow's wait container

`kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded`

gives `No resources found in argo namespace`.

`kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow} --context pro`

gives `No resources found in argo namespace`.

`kubectl get wf -o name -n argo --context pro | rg ${workflow}`

gives 

`kbrasil-4-1694509200            Running     20h`

workflow-controller log:

logs-from-workflow-controller-in-workflow-controller-54949c4496-2nrnn.log

[tczhao]

Hi @Guillermogsjc , Were you able to make any progress on this issue, or is it still reoccurring?

[Guillermogsjc]

Did not manage to find anymore.

Closing

[vadasambar]

Facing similar issue in v3.5.4. /reopen