ploomber / soopervisor

☁️ Export Ploomber pipelines to Kubernetes (Argo), Airflow, AWS Batch, SLURM, and Kubeflow.
https://soopervisor.readthedocs.io
Apache License 2.0
45 stars 18 forks source link

SLURM monitor progress #60

Closed edublancas closed 9 months ago

edublancas commented 2 years ago

Currently, the SLURM integration submits tasks but there is no built-in way to monitor them. An easy way to solve the problem would be to store a file with the job ids, then have a command check the status using the IDs