kubectl buildkit create --replicas N -> no ClusterIP service?

[mmisztal1980]

What steps did you take and what happened I'm setting up a multi-node build-farm within my k8s cluster using the following kubectl commands:

kubectl \
            --kubeconfig ./kubeconfig \
            --namespace build \
            create secret docker-registry buildkit \
            --docker-server=<REGISTRY_ADDRESS> \
            --docker-username=<USERNAME> \
            --docker-password=<PASSWORD> \
            --dry-run=client \
            -o yaml | kubectl apply --kubeconfig ./kubeconfig -f -

kubectl \
            buildkit create \
            --replicas <REPLICAS> \
            --namespace build \
            --kubeconfig ./kubeconfig

A desired amount of buildkit pods has been provisioned as expected.

What did you expect to happen I'd like to ensure that my set of pods is loadbalanced. Typically in k8s, one needs to create a ClusterIP service that will route all requests to the buildkit deployment. The buildkit k8s examples also mention this: https://github.com/moby/buildkit/tree/master/examples/kubernetes

• I'm looking for information what should the container port be? The example service manifests contain TCP port 1234, however I'm not sure if this is the actual port buildkit uses (the port is not exposed in the Dockerfile), and the buildkit cli docs also do not sufficiently explain this topic (not to mention that kubectl buildkit create does not create the said service).
• I'm assuming that, even If I create a multi-replica deployment, it is not loadbalanced due to the fact that the service is missing?
• I've noticed the podchooser package. Given my go knowledge is v.limited, upon 1st glance this package appears to choose a pod if multiple pods are running? Am I correct?

Can anyone share their insights?

[dhiltgen]

We do not expose buildkitd as a network accessible API in this tool, and instead rely on exec to access the builder running within the pod. You can find a more detailed description on how the communication works at https://github.com/vmware-tanzu/buildkit-cli-for-kubectl/blob/main/docs/architecture.md

When you run a build command with the CLI, it will select which builder to use in a multi-node cluster based on the pod chooser that was configured. The build will be performed on a single node (pod) and the image will then be replicated across to the other nodes after the build completes.