reanahub / reana-job-controller

REANA Job Controller
http://reana-job-controller.readthedocs.io/
MIT License
2 stars 38 forks source link

system: Introduce Kueue as an optional alternative approach how to submit user jobs #453

Open xaviertintin opened 6 months ago

xaviertintin commented 6 months ago

This pull request introduces changes to the reana-job-controller component to enable Kueue as an optional job submission method in REANA, working in conjunction with the previously submitted changes to reana-workflow-controller.

Key Changes:

  1. Environment Variable Access:

    • Retrieves the KUEUE_ENABLED environment variable within reana-job-controller/reana_job_controller/config.py. This variable reflects the admin's selection of Kueue during deployment via Helm values.
  2. Deployment Type Selection:

    • Based on the retrieved environment variable, the code in reana-job-controller/reana_job_controller/kubernetes_job_manager.py dynamically selects the appropriate job submission method:
    • Standard: If KUEUE_ENABLED is False, the existing Kubernetes Job API is used for job submission.
    • Kueue: If KUEUE_ENABLED is True, the pull request implements the necessary logic to utilize Kueue for job submission.

Benefits: