jhunt / k8s-boshrelease

A BOSH Release for deploying Kubernetes clusters
MIT License
13 stars 9 forks source link

Bind NodePort on loopback / 127.0.0.1/8 #29

Closed jhunt closed 4 years ago

jhunt commented 4 years ago

Eirini uses 127.0.0.1:32123 (or a different configurable NodePort) for images that it stages via buildpacks; this is the only endpoint a kubelet can natively access. Unfortunately, with ipvsadm, we are seeing that kube-proxy only binds the node IP for the NodePort traffic forwarding, so kubecf doesn't seem to work with ipvsadm.

Steps to Reproduce

Here's a daemonset that shows off the problem:

---
apiVersion: v1
kind: Namespace
metadata:
  name: gh-issue-29
---
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: gh-issue-29
  name:      www

data:
  index.html: |
    pong
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  namespace: gh-issue-29
  name:      github

spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx

          ports:
            - name: web
              containerPort: 80

          volumeMounts:
            - name: www
              mountPath: /usr/share/nginx/html

      volumes:
        - name: www
          configMap:
            name: www
---
apiVersion: v1
kind: Service
metadata:
  namespace: gh-issue-29
  name:      github
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
    - name: web
      port: 80
      targetPort: web

With this loaded, you should be able to curl http://<node-ip>:<node-port> and get back the text pong. If you then SSH onto one of the kubelets, and curl http://127.0.0.1:<node-port>, one of the following will happen:

  1. You get back pong, in which case the NodePort is bound on all interfaces and the bug is fixed.
  2. You get a connection refused, in which case the NodePort is only binding on the node IP, and the bug persists.

Fix the Kubernetes such that we get case 1, not case 2.