Restrict use of NodePorts

[gothub]

For our bare-metal k8s installation, we use an ingress controller that creates a NodePort.

It appears that behind the scene, k8s uses iptables to allow all traffic to all services that create NodePorts, independent of any ufw entered 'ALLOW' rules. The rational behind this may be to allow health checks.

On our dev k8s control node:

# iptables -L -v
Chain INPUT (policy DROP 76 packets, 3177 bytes)
 pkts bytes target     prot opt in     out     source               destination
 433M   72G cali-INPUT  all  --  any    any     anywhere             anywhere             /* cali:Cz_u1IQiXIMmKD4c */
 416M   69G KUBE-NODEPORTS  all  --  any    any     anywhere             anywhere             /* kubernetes health check service ports */
... [many lines omitted]
Chain KUBE-NODEPORTS (1 references)
 pkts bytes target     prot opt in     out     source               destination
... [many more lines omitted]

I believe that the ingress controller is secure, as URLs it doesn't know how to route are 'dumped' into a backend server that just displays a 404 error page.

So, we need to consider if making other services or UIs using NodePorts available on other ports is secure and desirable.

[ThomasThelen]

The more I read about NodePort the less I'm convinced it'll work within our constraints, primarily due to the insecurities with it (whitelisting IPs) and the number of hoops to jump though to try and remedy them. From what I can tell it looks like the LoadBalancer type would be better (if we have a load balancer) since we can set source IP ranges. If we can use the ingress controller in front of the NodePort to handle the restrictions, that might work.

[gothub]

@ThomasThelen the ingress controller we are using works with "ClusterIP" based services, IIRC, and these are not addressable outside the cluster's internal network, so rely on the ingress controller for routing.

If you have ideas on a different solution, please post here.

[ThomasThelen]

It sounds like we're cornered into using a NodePort because that's the only config option left which allows us to expose the service (and we can still cleanlist IPs with the ingress controller)

[ThomasThelen]

From the Jul 8 development call it's clear that I had a misunderstanding of ClusterIP. We should be using CluserIP instead of NodePort. I created a relevant issue here and have an open PR that I'll open once this is tested.