search

actions / runner-container-hooks

Runner Container Hooks for GitHub Actions
MIT License
63 stars 41 forks source link

Make _work PVC optional for k8s job containers #129

Closed cmmarslender closed 3 months ago

cmmarslender commented 6 months ago

I'm working on updating to the latest ARC in our cluster, and its all working, but one thing we wanted to do was just use emptyDir for the _work volume, vs having to use a PVC. We have a few options for using PVCs, but since they are ephemeral and get deleted when the pods go away, we'd prefer to just use emptyDir.

In https://github.com/actions/runner-container-hooks/blob/main/packages/k8s/src/k8s/index.ts#L101 it looks like the default PVC is hard coded. Later in the file, its possible to extend that from extension.spec but that only ends up merging the volume list - doesn't do anything to override the _work volume that already exists.

There's probably a few ways to go about accomplishing this, but probably the easiest from a user perspective is just to set emptyDir: true or similar when deploying the auto scaling runner sets in ARC, and the value can be passed along and end up in the hooks here, where the setting can be implemented? (alternatively, maybe just something like workVolume.enabled: true/false ?

rteeling-evernorth commented 4 months ago

I'm pretty sure they use a PVC instead of an emptydir because the runner pod, the workflow pod and any action job pods are all separate pods instead of one (at least in kubernetes mode). EmptyDir is only available to share files within the context of a single pod, it cannot be shared between more than one.

https://kubernetes.io/docs/concepts/storage/volumes/#emptydir

cmmarslender commented 4 months ago

I'm pretty sure they use a PVC instead of an emptydir because the runner pod, the workflow pod and any action job pods are all separate pods instead of one (at least in kubernetes mode). EmptyDir is only available to share files within the context of a single pod, it cannot be shared between more than one.

kubernetes.io/docs/concepts/storage/volumes/#emptydir

I'll have to go see what its attempting to share between different pods and why. Depending on the volume provisioner in use, sharing PVCs between different pods may not even work. RWX or else landing on the same underlying host is usually going to be required for that to even work.

nikola-jokic commented 3 months ago

Hey everyone! @rteeling-evernorth is 100% right. We have to use PVCs to share workflow data between newly created pods. I'm closing this one since it is not possible to use emptydir to share data between multiple pods.