Update congress to lock down all namespaces

[whitlockjc]

Congress currently only manages Shipyard-created namespaces and unfortunately, this means it is all to easy to create or grandfather in namespaces that are insecure. For a completely secure multi-tenant Kubernetes environment, we should update congress to watch all namespaces and lock them down accordingly.

What does locking things down mean? These are the three main rules that happen via congress:

• Deny all traffic to/from the namespace by default
• Open up all traffic within the namespace
• Open up all traffic from the k8s-routers in the apigee namespace

Of course, these rules don't apply to all namespaces so we need a way to ignore and disable congress. To do this, we should have a configurable list of namespaces to never touch, like kube-system. We also should have a label-based approach to turn off management. (One concern with this last bit is we don't want tenants to be able to turn off their security so we likely don't want to implement this until authz is enabled disabling such an action. Or we could just make it so that congress disallows this?)

/cc @jlin21, @madhurranjan

[noahdietz]

I have the implementation already. What I need is a confirmed list of namespaces to exclude from network isolation.

My exclusion list right now is: "kube-system,apigee,apigee-jenkins-prod,calico-system" Here is a list of non-shipyard created tenants (other apigee engineers in the cluster) in e2e: "desired-state,guardians,monitoring,monitoring-e2e-test"...they will be locked down unless specifically excluded. This means that the router will be able to talk to them, but no other namespace can communicate with them like Jeremy mentioned above. Is that ok, @madhurranjan @jlin21 ?

[noahdietz]

Also, if there is a label you want to use for toggling exclusion (i.e. "congress=exclude"), any ideas are welcome

[jlin21]

The label with congress=exclude is a good idea when the auth portion is in place. We can discuss the exclusion list later, but the keys ones for every k8s cluster will be the ones you listed above, minus apigee-jenkins-prod. Other than that I agree with all comments you guys posted. As @madhurranjan would say "ack"

[noahdietz]

I've locked down e2e in the ways mentioned above. There were some hiccups, but they've been overcome.

Going to wait a few hours to allow for congress restart to occur, ensuring that works, before moving to prod

[noahdietz]

Locked down prod. Closing.