Implement NetworkPolicy

[minrk]

To better lockdown the internal network, we should start instituting NetworkPolicies.

In particular, these allow us to set ingress and egress policies for which pods should have access to what. Egress policy support requires calico 2.6.1, which should be new in the latest GKE release (1.8.4-gke.0).

Starting this issue for sketching out what our inter-connect network needs really are. For various services:

user pods:

• egress:
  • world, only whitelisted ports (http/s, git)
  • except cluster IPs
• ingress:
  • proxy
  • hub

builds (docker-in-docker, user build code):

• egress:
  • world, only whitelisted ports (http/s, git)
  • docker registry (for now, fits above, but could be local in the future)
  • except cluster IPs
• ingress: None (using mounted socket, no http)

binder:

• egress:
  • kubernetes itself (to launch build)
  • jupyterhub proxy (or ingress ip/host?)
  • world, only whitelisted ports (http/s) for resolving refs
• ingress: nginx-ingress only

hub:

• egress:
  • proxy
  • user pods
• ingress:
  • proxy
  • user pods (if using jupyterhub auth, not with what we do now)

What about data collection services, such as prometheus?

We should figure out the best way to express these requirements, especially in a way that allows deployments to add additional services permissions. That probably means using label-selectors to allow access to pods, and making sure that those label selectors are easy to apply where they are needed.

[consideRatio]

The Z2JH network policy system is quite thorough and tested at this point. I suggest we mimic the configuration options from that Helm chart. In practice we should define a NetworkPolicy for each of our BinderHub specific workloads - the binderhub pods and the build pods I think.

For details about the z2jh implementation the following documentation and such is relevant:

1. A NetworkPolicy definition, we can inspect the hub pod's netpol resource
2. Documentation about the netpol configuration, which is very similar for all netpols in z2jh's workloads (hub, proxy.chp - proxy pod, proxy.traefik -autohttps pod, scheduling.userScheduler - user-scheduler pod).