Document/examples for additional ingress gateways / non-HTTP traffic

[mjnagel]

UDS Core and the UDS Operator out of the box only support HTTP traffic. In some scenarios it may be necessary/desirable to expose a non-HTTP service through gateways.

To do this an end user needs to:

• Provision a new ingress gateway with the necessary ports exposed/forwarded (using the upstream istio helm chart) OR override the ports exposed on one of the existing gateways (tenant/admin)
• Create a Gateway custom resource selecting that ingress gateway, that also specifies the necessary port
• Create a VirtualService with the necessary spec (commonly tcp)
• Create a NetworkPolicy to allow traffic between all the pieces (ingressgateway -> service)

One example of this is Gitlab/SSH:

• Extra manifests: https://github.com/defenseunicorns/uds-bundle-software-factory-nutanix/tree/main/packages/additional-manifests/gitlab
• Bundle overrides to add port to ingress gateway: https://github.com/defenseunicorns/uds-bundle-software-factory-nutanix/blob/main/bundles/uds-core-swf/uds-bundle.yaml#L346-L366

We would like to provide a doc explaining how to do this (separate zarf package with these pieces most likely) and potentially an example of what this would look like. We should also add a warning about why this is not directly enabled in Core and should only be used if required. Primary reasons include:

• simplifying the network boundary, "everything goes through an HTTPS Istio ingress gateway" (allowing non-HTTPs through istio is quite different)
• reducing attack vectors (non-http/s ports may expose additional vulnerabilities)

[mjnagel]

This may also help with https://github.com/defenseunicorns/uds-core/issues/785