docs: How to deploy Jitsi (and potentially other examples) into DOKS

[davidkornel]

This issue aims to document how to deploy the Jitsi example into a Digital Ocean Kubernetes cluster.

Jitsi

The mentioned example/tutorial was created using GKE, which means it wasn't tested on other cloud providers. Unfortunately, DOKS is much more strict about creating load balancer services (with a public IP address). To expose TCP ports to the public internet is easy, and there is nothing to modify, however, to expose UDP ports requires some fine-tuning. If a load balancer uses UDP in its forwarding rules, the load balancer requires that a health check port is set that uses TCP, HTTP, or HTTPS to work properly (DOKS health check).

The most important fact is that a health check port must be exposed to the public internet, just to get the load balancer up and running. This is not too secure, because this port is unprotected and lets anyone test this port and get information about the health of your pods in the cluster. While it's unfortunate it is a must-have configuration.

In order to achieve a working UDP load balancer a slightly modified GatewayConfig and Gateway must be used. loadBalancerServiceAnnotations will be added to the created service as extra annotations. These will tell the DOKS API where and how to check the healthiness of the underlying endpoints (pods). And an extra TCP port on 8086 will be exposed used for health checking.

apiVersion: stunner.l7mp.io/v1alpha1
kind: GatewayConfig
metadata:
  name: stunner-gatewayconfig
  namespace: stunner
spec:
  authType: longterm
  sharedSecret: "my-shared-secret"
  loadBalancerServiceAnnotations:
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-port: "8086"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-protocol: "http"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-path: "/live"
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: Gateway
metadata:
  name: udp-gateway
  namespace: stunner
spec:
  gatewayClassName: stunner-gatewayclass
  listeners:
    - name: health-check
      port: 8086
      protocol: TCP
    - name: udp-listener
      port: 3478
      protocol: UDP

What about other media servers, such as LiveKit?

Haven't tested yet but other examples should work the same way.

• UDP load balancer services must have an extra health check port
• GatewayConfig should have loadBalancerServiceAnnotations added to their configs
• Gateway must have an extra health check port

[rg0now]

Nice, thanks. What about adding a new subsection to the Jitsi tutorial README, something along the lines of "Enable health-checking" or similar?

Some notes:

• With the new (yet unreleased) version of STUNner you can add per-Gateway annotations: any annotation added to the Gateway will be automatically added to the LB service created by the STUNner operator to expose the Gateway. Gateway annotations override GatewayConfig.Spec.LoadBalancerServiceAnnotations on conflict.

apiVersion: stunner.l7mp.io/v1alpha1
kind: GatewayConfig
metadata:
  name: stunner-gatewayconfig
  namespace: stunner
spec:
  authType: longterm
  sharedSecret: "my-shared-secret"
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: Gateway
metadata:
  name: udp-gateway
  namespace: stunner
  annotations:
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-port: "8086"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-protocol: "http"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-path: "/live"
spec:
  gatewayClassName: stunner-gatewayclass
  listeners:
    - name: health-check
      port: 8086
      protocol: TCP
    - name: udp-listener
      port: 3478
      protocol: UDP

• Multiprotocol LB services are not supported as of now, since support is fairly uneven across Kubernetes versions. The STUNner operator will remove the TCP service-port from the LB service it creates, leaving only the UDP port in. This means you will have to create the LB service to expose the Gateway manually: just copy the Service created by STUNner and add the missing service-ports.

• In order to avoid having two LB Services for the Gateway (one created automatically and the manual copy), which usually costs extra $$$, we can fix the type of the service created by STUNner to ClusterIP by adding the special annotation stunner.l7mp.io/service-type: ClusterIP to the Gateway (or to the GatewayConfig loadBalancerServiceAnnotations but that will affect all services created by STUNner). This will still create a service but the type will be ClusterIP so it will not be exposed to the Internet, and this usually does not cause extra $$$.

apiVersion: gateway.networking.k8s.io/v1alpha2
kind: Gateway
metadata:
  name: udp-gateway
  namespace: stunner
  annotations:
    stunner.l7mp.io/service-type: ClusterIP
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-port: "8086"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-protocol: "http"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-path: "/live"
spec:
  gatewayClassName: stunner-gatewayclass
  listeners:
    - name: health-check
      port: 8086
      protocol: TCP
    - name: udp-listener
      port: 3478
      protocol: UDP

[rg0now]

Closing this due to inactivity. Feel free to reopen if the problem persists.