[BUG] A failed wf node leaves the other nodes (spark tasks) running until they finish

[EngHabu]

Describe the bug If a node in a workflow fails, the entire workflow fails, if there are other nodes running spark, the underlying CRD will continue to run wasting resources.

Expected behavior If the workflow shows as failed, the underlying CRDs it created should be cleaned up.

Flyte component

• [ ] Overall
• [ ] Flyte Setup and Installation scripts
• [ ] Flyte Documentation
• [ ] Flyte communication (slack/email etc)
• [X] FlytePropeller
• [ ] FlyteIDL (Flyte specification language)
• [ ] Flytekit (Python SDK)
• [ ] FlyteAdmin (Control Plane service)
• [ ] FlytePlugins
• [ ] DataCatalog
• [ ] FlyteStdlib (common libraries)
• [ ] FlyteConsole (UI)
• [ ] Other

Environment Flyte component

• [ ] Sandbox (local or on one machine)
• [X] Cloud hosted
  • [X] AWS
  • [ ] GCP
  • [ ] Azure
• [ ] Baremetal
• [ ] Other

[kumare3]

@EngHabu does the abort not propagate?