Closed tmjd closed 7 years ago
$ ./calicoctl get policy -o yaml
- apiVersion: v1
kind: policy
metadata:
name: advanced-policy-demo.allow-dns
spec:
egress:
- action: allow
destination:
ports:
- 53
selector: calico/k8s_ns == 'kube-system' && k8s-app == 'kube-dns'
protocol: udp
source: {}
order: 400
selector: calico/k8s_ns == 'advanced-policy-demo'
types:
- egress
- apiVersion: v1
kind: policy
metadata:
name: advanced-policy-demo.deny-egress
spec:
egress:
- action: deny
destination:
notSelector: calico/k8s_ns == 'advanced-policy-demo'
source: {}
order: 500
selector: calico/k8s_ns == 'advanced-policy-demo'
types:
- egress
- apiVersion: v1
kind: policy
metadata:
name: knp.default.advanced-policy-demo.access-nginx
spec:
egress:
- action: allow
destination: {}
source: {}
ingress:
- action: allow
destination: {}
source:
selector: calico/k8s_ns == 'advanced-policy-demo'
order: 1000
selector: calico/k8s_ns == 'advanced-policy-demo' && run == 'nginx'
types:
- ingress
- apiVersion: v1
kind: policy
metadata:
name: knp.default.advanced-policy-demo.default-deny
spec:
egress:
- action: allow
destination: {}
source: {}
order: 1000
selector: calico/k8s_ns == 'advanced-policy-demo'
types:
- ingress
- apiVersion: v1
kind: policy
metadata:
name: knp.default.policy-demo.access-nginx
spec:
egress:
- action: allow
destination: {}
source: {}
ingress:
- action: allow
destination: {}
source:
selector: calico/k8s_ns == 'policy-demo' && run == 'access'
order: 1000
selector: calico/k8s_ns == 'policy-demo' && run == 'nginx'
types:
- ingress
- apiVersion: v1
kind: policy
metadata:
name: knp.default.policy-demo.default-deny
spec:
egress:
- action: allow
destination: {}
source: {}
order: 1000
selector: calico/k8s_ns == 'policy-demo'
types:
- ingress
This seems to be the iptables rule that is dropping the traffic
[2:120] -A cali-po-_l2a5_DDU8iKXgoc8Dav -m comment --comment "cali:WXDWSDmbDYSTHADG" -m set ! --match-set cali4-s:uK4R6n61sYuSXSz0NO3eBIb dst -j DROP
Note this was when the 'access' pod was on the same host as the destination pod which was specified by the nginx pod's IP address and both pods were running on the same host.
I think this is occurring because of the new types
field and how it behaves.
In previous versions of the demo, the policy controller programmed egress "allow" rules which allowed all egress traffic. That's why the demo included a "deny all traffic leaving the namespace" rule to override that behavior.
Given that those egress rules are no longer taking affect because of the types: [ingress]
part of the policy, the "deny all traffic leaving the namespace" rule should probably instead be a allow traffic within the namespace
rule.
e.g.
- apiVersion: v1
kind: policy
metadata:
name: advanced-policy-demo.deny-egress
spec:
egress:
- action: allow
destination:
selector: calico/k8s_ns == 'advanced-policy-demo'
source: {}
order: 500
selector: calico/k8s_ns == 'advanced-policy-demo'
types:
- egress
@tmjd could you try that instead and see if the demo behaves as expected?
So switching the 'deny-egress' policy to that does seem to work. I can now reach the nginx service but cannot ping google.com
.
We'll need to release note this behavior change in big red letters.
This is the right behavior - what was there before was a workaround for missing Ingress/Egress types support.
Actions:
@tmjd are you OK handling the latter?
Fixed by #1133
With the pre-release version of Calico v2.6.0 when testing the Advanced policy tutorial, the step "Prevent outgoing connections from pods" blocked all outgoing traffic from my access pod. https://docs.projectcalico.org/v2.6/getting-started/kubernetes/tutorials/advanced-policy#prevent-outgoing-connections-from-pods
Expected Behavior
All steps of the tutorial should work.
Current Behavior
The step attempting to block outgoing DNS traffic blocked all outgoing traffic. I even attempted to use the service IP for nginx and the IP of the nginx pod that lived on the same host as the access pod.
Possible Solution
Fix it :smile:
Steps to Reproduce (for bugs)
Context
I saw this when using my Vagrant cluster set up with etcd TLS that I had previously upgraded from v2.5.1. I don't believe the upgrade impacted this but wanted to mention it incase it did.
Your Environment
Calico version:
Orchestrator version (e.g. kubernetes, mesos, rkt): K8s v1.7.5
Operating System and version: Container Linux 1465.7.0