search

dask / dask-kubernetes

Native Kubernetes integration for Dask
https://kubernetes.dask.org
BSD 3-Clause "New" or "Revised" License
313 stars 148 forks source link

Allow setting generateName in DaskJob resources #896

Open msyulia opened 5 months ago

msyulia commented 5 months ago

What?

Setting metadata.generateName in kubernetes jobs (batch/v1 Job) allows to re-run jobs after applying the resource again without the need to delete the previous job [source]. Currently trying to set this field on a DaskJob resource results in missing metadata.name in object {kubernetes.dask.org/v1 DaskJob}

Why?

Having the name be randomised via generateName on a DaskJob resource would allow users to rerun those jobs without the need to delete the completed jobs. In my use case I would like to deploy DaskJobs using ArgoCD as the k8s controller, generating the name on the fly would allow to rerun the job whenever a sync is triggered.

How?

Following the ObjectMeta spec:

apiVersion: kubernetes.dask.org/v1
kind: DaskJob
metadata:
    generateName: dask-hello-world-

Imo I don't see a reason to implement this feature in the python library, but maybe someone will prove me wrong 😄

As a sidenote, I'd be happy to work on this if determined to be useful.

jacobtomlinson commented 5 months ago

This sounds like a great improvement. If you're happy to work on this that would be awesome.

I agree that we probably don't need to worry about the Python client.