Separate control and data planes

[f5yacobucci]

As a route to efficacy and quickly understanding the Gateway API; its implementation and alignment to NGINX as a data plane, we decided on a simplified, but rigid, deployment pattern. To improve our security posture and installation flexibility the control and data planes should be separated as semi-autonomous, distributed components.

Problem Control plane containers and data plane containers compose into a single Kubernetes Pod. Control plane containers use OS signals and file system sharing to exchange data. Control plane and data plane are governed by the same RBAC policies as they reside in one Pod and ServiceAccount. Control plane and data plane must scale dependently and cannot scale on independent axis. Compromise of the control plane may impact customer traffic in the data plane. Compromise of the data plane may expose Kubernetes API server and impact the cluster and allow horizontal movement in the network. Kubernetes secrets and sensitive data will be shared across containers unnecessarily. Violation of a basic zero-trust tenet: "The data plane and control plane are logically separated." NIST SP 800-207

Solution Separate control and data planes across independent Pods and Deployments.

A/C:

Create design document outlining new architecture and deployment pattern. Should cover communication channels between control and data planes, authentication and authorization, interfaces, and data plane agent options. Both planes should eventually scale, only data plane is required for the first iteration.

• Deployment architecture
• Communication channels / protocols
• Authentication and authorization
• Data plane registration and scaling

### Stories
- [x] #344
- [x] #376 
- [x] #377 
- [ ] #378 
- [ ] #379
- [ ] #375   
- [ ] #380 
- [ ] #381 
- [ ] #374 
- [ ] #382 
- [ ] #459

Out of scope:

• Control plane scaling is a non-goal for this iteration. Following work will design HA, leader election, sharding or other techniques based on requirements.
• Explicit support for multiple control planes per gateway class or namespace segmentation. Following work will highlight tenant and other isolation use cases.

Aha! Link: https://nginx.aha.io/features/NKG-72

[mpstefan]

I think before we pick this back up, we may want do two things:

1. Define the value to the user of this new architecture
2. Define the architecture we want to achieve with this change, or over time

[mpstefan]

Mistakenly marked as complete. Moving this functionality to a new epic.