Allow Gateways to request a specific NodePort in the automatically created Service

[rg0now]

Several users requested for a way to enforce a specific NodePort when exposing a Gateway. This can be done by, say, adding the annotation stunner.l7mp.io/nodeport: <my-nodeport> that would the operator consider when creating the Service. Enforcing a NodePort that is already used is an error.

This issue is to track the progress in implementing this feature.

[smeng9]

We also need to consider the case of complex-gateway with multiple listeners. Then we probably need multiple node ports.

[rg0now]

Hmm, very good point, thx! What about a list of key-value pairs in the annotation value like stunner.l7mp.io/nodeport: ["udp-listener":32000,"tcp-listener":32001,"tls-listener":32002,"dtls-listener":32003], where each key-value pair is a name of a listener and a requested nodeport? Wdyt?

[smeng9]

Sounds good to me

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Gabor Retvari @.> Sent: Wednesday, March 27, 2024 1:06:50 AM To: l7mp/stunner @.> Cc: Meng, Shaoyu @.>; Comment @.> Subject: Re: [l7mp/stunner] Allow Gateways to request a specific NodePort in the automatically created Service (Issue #137)

Hmm, very good point, thx! What about a list of key-value pairs in the annotation value like stunner.l7mp.io/nodeport: ["udp-listener":32000,"tcp-listener":32001,"tls-listener":32002,"dtls-listener":32003], where each key-value pair is a name of a listener and a requested nodeport? Wdyt?

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/l7mp/stunner/issues/137*issuecomment-2021005187__;Iw!!DZ3fjg!8VO2tUkOMbph9bWfEz3eWci2cAb82H3mvnGt2R2j2jDGkKVhhoAcZK1m4n6pWbF4z3axBtzjpkc-zOxWK7Sgk-lejkU$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AJHAEC372BUFUWKWS2ZE3RDY2GMJVAVCNFSM6AAAAABFJFMRMGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRRGAYDKMJYG4__;!!DZ3fjg!8VO2tUkOMbph9bWfEz3eWci2cAb82H3mvnGt2R2j2jDGkKVhhoAcZK1m4n6pWbF4z3axBtzjpkc-zOxWK7SgwcYYRQU$. You are receiving this because you commented.Message ID: @.***>

[rg0now]

This is now implemented in https://github.com/l7mp/stunner-gateway-operator/commit/b92ef6a99766c5ac0b3fadcf94aca8e2bb9f55f0, you can install it from the dev channel. Dear @smeng9, can you please test this and report any problem you find?

[smeng9]

Hi @rg0now sorry for late reply

I have performed some test and seems there are some issues creating the service/complex-gateway in 0.19.0

Here is the list of what kubernetes resources originally I had in 0.18.0 . In 0.18.0 I manually used kube edit to change the node port number of service/complex-gateway after a deploy.

I used the annotation in 0.19.0 but I cannot find the service/complex-gateway anymore

Here is my config. I realized there are some changes are not very clear in the doc, first stunner.l7mp.io/enable-mixed-protocol-lb type seems gets changed from boolean to a string. Second I have to add escapes to the stunner.l7mp.io/nodeport.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: complex-gateway
  namespace: stunner
  annotations:
    stunner.l7mp.io/enable-mixed-protocol-lb: "true"
    stunner.l7mp.io/nodeport: "{\"udp-listener\":31478,\"tcp-listener\":31479}"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-port: "8086"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-protocol: "http"
    service.beta.kubernetes.io/do-loadbalancer-healthcheck-path: "/live"
spec:
  gatewayClassName: stunner-gatewayclass
  listeners:
    - name: udp-listener
      port: 31478
      protocol: TURN-UDP
      allowedRoutes:
        namespaces:
          from: All
    - name: tcp-listener
      port: 31479
      protocol: TURN-tcp
      allowedRoutes:
        namespaces:
          from: All

Is there way I can troubleshoot the service/complex-gateway not showing up?

[smeng9]

Please ignore previous message. Ah finally I used a completely clean setup seems the annotation is successfully applied. Maybe due to some outdated CRDs when upgrading.

[rg0now]

Unfortunately, if there's a nodeport conflict (some other Service is already using the same nodeport) then the Service will fail to update and you won't see any Service at all. That's one of the reasons we advise against using NodePorts. Happy you made it work!