search

dask / community

For general discussion and community planning. Discussion issues welcome.
19 stars 3 forks source link

Extract generic Kubernetes library code from dask-kubernetes into a new project #313

Closed jacobtomlinson closed 1 year ago

jacobtomlinson commented 1 year ago

Hey folks. I wanted to open this issue on the community tracker to get a little more visibility than opening it just in dask-kubernetes.

In dask-kubernetes we use a combination of kubernetes-asyncio and pykube-ng (and kubectl in a subprocess) to interact with the Kubernetes API. We have various shims and wrappers on these to plug some holes, like GKE auth.

For better code reuse and maintenance I'd like to extract this code into a new project. Ultimately I want to build this into a new Kubernetes Python library that has all of the features we need for dask-kubernetes and beyond. I have thoughts about the API design of the existing libraries and would take a different approach myself.

I don't think the Dask org (or dask-contrib) are the right place for this new repo to live. So I was thinking about doing it either in my own account or creating a new org for it to live in. All of the existing code in dask-kubernetes is BSD-3 so my assumption is that as long as the new repo is also BSD-3 licensed folks would be fine with this. But I wanted to open a discussion here in case folks have thoughts.

martindurant commented 1 year ago

I'm expect no one will have a problem with splitting out the functionality, and with BSD3, certainly no-one can stop you. However, I would think that dask-contrib is a perfectly fine place to put it, since it would a "a library useful to dask". If you want other orgs to put it in, fsspec and intake would be happy to, even if it doesn't exactly align with what those orgs do.

I would tentatively say that using your own account gives a project a brand of "alpha" software, and I think also changes some of the rules around forks and contributing. Maybe none of that is very important.

jacobtomlinson commented 1 year ago

I went with a new org/repo https://github.com/kr8s-org/kr8s.

Keep your eyes peeled over the coming weeks/months for an awesome new Kubernetes Python library 😃 .