kubeflow / mpi-operator

Kubernetes Operator for MPI-based applications (distributed training, HPC, etc.)
https://www.kubeflow.org/docs/components/training/mpi/
Apache License 2.0
417 stars 209 forks source link

"cleanPodPolicy: All" does not clean up launcher pod #643

Open hy00nc opened 1 month ago

hy00nc commented 1 month ago

In MPIJob spec.runPolicy.cleanPodPolicy, we can set "All" to clean up all pods after completion including the launcher pod. However, it seems it only cleans up worker pods, as you can see in this code.

Using the official example pi.yaml, if you change "cleanPodPolicy: Running" to "cleanPodPolicy: All", you can reproduce the issue.

alculquicondor commented 1 month ago

Feel free to open a PR