cilium/istio followup

[jefflill]

Here are some things to follow up on for the cilium/istio changes:

• [ ] evaluate using the Istio Telemetry resource
• [ ] configure resource limits from cluster advice
  • [ ] cilium components
  • [ ] istio components
• [ ] pod placement
  • [ ] istio-ingressgateway on all nodes?
  • [ ] istiod on all control-plane nodes
  • [ ] cilium-operator on all control-plane nodes
  • [ ] use topologySpreadConstraints so the deployments above to match the number of target nodes or have neon-cluster-operator watch nodes and adjust the replica count? We'll also need to configure anti-affinity.
• [ ] istio-ingressgateway: configure the ingress rules
• [ ] Convert to standard V1Gateway for ingress specifications? This appears to handle UDP which Istio doesn't appear to support yet.
• [ ] Add PriorityClassName to network services:
  • [ ] istio-cni
  • [ ] istio-ingress-gateway
  • [ ] istiod
  • [ ] cilium
  • [ ] cilium-operator