Support kaniko - incluster builds

jlewi / hydros

hydros automates hydrating and committing configuration

Apache License 2.0

4 stars 0 forks source link

Support kaniko - incluster builds #78

Open jlewi opened 5 months ago

jlewi commented 5 months ago

Now that we have code for building the context launching as a Job running kaniko in K8s should be relatively straightforward.

Possible advantages of using K8s

Pools of available resources
More resources (in particular RAM) than what's available in GCB

This PR has some YAML for launching kaniko on k8s https://github.com/jlewi/notes/pull/12

jlewi commented 5 months ago

Not sure how useful this would be. It looks like kaniko has a dependency on ephmeral-storage https://github.com/GoogleContainerTools/kaniko/issues/2219. I think on GKE clusters this maxes out at 10Gi on standard clusters and 1Gi on Autopilot clusters. This turns out to be too low for really big images like CUDA.