search

kubernetes-sigs / cluster-api-provider-openstack

Cluster API implementation for OpenStack
https://cluster-api-openstack.sigs.k8s.io/
Apache License 2.0
289 stars 252 forks source link

Nodeports open from 0.0.0.0/0 by default, causing unexpected behaviour together with Floating IPs #2127

Open huxcrux opened 3 months ago

huxcrux commented 3 months ago

/kind bug

What steps did you take and what happened: Nodeports are by default open from 0.0.0.0/0. This has historically not been a problem since nodes previously had an internal IP and was nat:ed over the Openstack router.

However when using Floating IPs this causes nodeports to be exposed from internet. This could lead to some unexpected behaviour. And sadly most end-users does not configure network policies properly meaning some services by mistake are accessible from the internet.

Since nodeport rules are added by default and there is no way of opting out this behaviour cannot be changed by the cluster admin.

Rules are added here: https://github.com/kubernetes-sigs/cluster-api-provider-openstack/blob/main/pkg/cloud/services/networking/securitygroups.go#L203 And defined here: https://github.com/kubernetes-sigs/cluster-api-provider-openstack/blob/main/pkg/cloud/services/networking/securitygroups_rules.go#L144C1-L165C1

I asked in slack if anyone was aware of any reason for nodeports to be open from 0.0.0.0/0 and got a reply saying it was due to conformance tests.

I think the best case of action would be to only allow nodeports from cluster subnet(s).

What did you expect to happen: Nodeports to not be open from internet without explicitly configuring it.

Anything else you would like to add: We may also want to change how rules for the API alongside AdditionalPorts exposed over the API Loadbalancer. However I would say nodeports are more "critical" since I see no reason in having Floating IPs on control-plane node at this time.

I'm currently running a conformance test with the following "change" applied to se if it passes: https://github.com/kubernetes-sigs/cluster-api-provider-openstack/commit/1685c65c303dbcf4bae2959705793c3133bd97fd

Environment:

mdbooth commented 3 months ago

@EmilienM I'm thinking that the default rules for NodePorts should probably restrict incoming traffic to workers and control plane. Anybody who wants to explicitly expose NodePorts externally with a FIP could add another rule to do that. What do you think?

EmilienM commented 3 months ago

@EmilienM I'm thinking that the default rules for NodePorts should probably restrict incoming traffic to workers and control plane. Anybody who wants to explicitly expose NodePorts externally with a FIP could add another rule to do that. What do you think?

works for me. @huxcrux are you going to submit a PR? We'll need doc as well.

huxcrux commented 3 months ago

@EmilienM I'm thinking that the default rules for NodePorts should probably restrict incoming traffic to workers and control plane. Anybody who wants to explicitly expose NodePorts externally with a FIP could add another rule to do that. What do you think?

works for me. @huxcrux are you going to submit a PR? We'll need doc as well.

Sure thing, just need to make sure the change passes conformance tests

huxcrux commented 3 months ago

Looks like conformance passed. Starting to work on a PR :)

Plugin: e2e
Status: passed
Total: 7411
Passed: 392
Failed: 0
Skipped: 7019

Plugin: systemd-logs
Status: passed
Total: 6
Passed: 6
Failed: 0
Skipped: 0
k8s-triage-robot commented 1 week ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale