cnoe-io / idpbuilder

Spin up a complete internal developer platform with only Docker required as a dependency.
https://cloud-native.slack.com/archives/C05TN9WFN5S
Apache License 2.0
200 stars 59 forks source link

[Bug]: If using `--kind-config` it must include networking config #367

Closed abangser closed 3 months ago

abangser commented 3 months ago

What is your environment, configuration, and command?

Environment: Local

Configuration:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraMounts:
      - containerPath: /var/lib/kubelet/config.json
        hostPath: /Users/abbybangser/.docker/config.json

Command:

$ ./idpbuilder create --kind-config ./kind-platform-config.yaml

What did you do and What did you see instead?

I tried to use kind config flag, but in doing so seemed to remove some key features that are auto populated otherwise. Namely the extraPortMapping which provides the networking between services and likely the kubeadmConfigPatches and containerdConfigPatches config is necessary too for the networking.

I would ideally see these default values still used, but even just documenting what is necessary if you do provide your own config could work for now.

Additional Information. Logs.

idpbuilder version:

$ ./idpbuilder version
idpbuilder 0.7.0-nightly.20240815 go1.21.3 darwin/arm64

Full kind config:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraMounts:
      - containerPath: /var/lib/kubelet/config.json
        hostPath: /Users/abbybangser/.docker/config.json

Top of logs for my run:

$ ./idpbuilder create --kind-config ./kind-platform-config.yaml
time=2024-08-26T20:57:59.620+01:00 level=INFO msg="Creating kind cluster" logger=setup
time=2024-08-26T20:57:59.632+01:00 level=INFO msg="Runtime detected" logger=setup provider=docker
########################### Our kind config ############################
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    image: kindest/node:v1.27.3
    extraMounts:
      - containerPath: /var/lib/kubelet/config.json
        hostPath: /Users/abbybangser/.docker/config.json

#########################   config end    ############################

Top of logs for a run without kind config:

$ ./idpbuilder create --kube-version v1.27.3
time=2024-08-26T20:55:37.619+01:00 level=INFO msg="Creating kind cluster" logger=setup
time=2024-08-26T20:55:37.635+01:00 level=INFO msg="Runtime detected" logger=setup provider=docker
########################### Our kind config ############################
# Kind kubernetes release images https://github.com/kubernetes-sigs/kind/releases
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  image: "kindest/node:v1.27.3"
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 443
    hostPort: 8443
    protocol: TCP

containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."gitea.cnoe.localtest.me:8443"]
    endpoint = ["https://gitea.cnoe.localtest.me"]
  [plugins."io.containerd.grpc.v1.cri".registry.configs."gitea.cnoe.localtest.me".tls]
    insecure_skip_verify = true

#########################   config end    ############################
nabuskey commented 3 months ago

Thanks for reporting this. I just opened a PR to address the extra port mapping part. https://github.com/cnoe-io/idpbuilder/pull/368

I'd say kubeadmConfigPatches is also needed since this is for the ingress-nginx controller. We should fix this.