search

Kong / kubernetes-ingress-controller

:gorilla: Kong for Kubernetes: The official Ingress Controller for Kubernetes.
https://docs.konghq.com/kubernetes-ingress-controller/
Apache License 2.0
2.22k stars 593 forks source link

Document how to support multiple GatewayClass instances #5237

Open mheap opened 11 months ago

mheap commented 11 months ago

Is there an existing issue for this?

Problem Statement

From @battlebyte:

Following the example in the GatewayClass documentation about creating two GatewayClass resources for Internet facing traffic and private facing traffic, we could create two GatewayClass definitions like the following:

kind: GatewayClass
metadata:
  name: internet
spec:
  controllerName: "konghq.com/internet-controller"
---
kind: GatewayClass
metadata:
  name: private
spec:
  controllerName: "konghq.com/private-controller"

I deploy two different instances of KIC and I want one instance to process “internet” manifests and the other instance “private” manifests. The GatewayClass specification says that the GatewayClass.spec.controller field determines the controller implementation responsible for managing the GatewayClass . What parameter/value do I need to use in the Helm chart to link a KIC deployment to konghq.com/internet-controller and the other deployment to konghq.com/private-controller ?

And a follow up question: is it possible for one single KIC deployment to be the implementation for multiple GatewayClasses?

Proposed Solution

From @czeslavo:

You can set ingressController.env.gateway_api_controller_name to konghq.com/internet-controller) or konghq.com/private-controller to make KIC reconcile the GatewayClasses you've created.

Currently, we only support a single GatewayClass per KIC instance (analogically to a single IngressClass).

Additional information

Where does this live on docs.konghq.com? Maybe under Guides->Security?

Acceptance Criteria

rainest commented 11 months ago

And a follow up question: is it possible for one single KIC deployment to be the implementation for multiple GatewayClasses?

Not for the use case proposed. Kong binds all routing configuration to all proxy listens equally. If you configure an internal address and external address for proxy_listen, all your routes will be accessible at either address.

Although we have source and destination IP matching configuration for routes, this is only available for stream subsystem plain TCP and UDP routes. Adding support for this in the http subsystem would probably be insufficient, as you'd still have all kong.conf-level configuration associated with a single proxy listen. The ip-restriction plugin is also a poor candidate to hack in support, as doing so would prevent users from configuring their own for any restrictions beyond "entire LAN network".

Network restriction concerns aside, supporting multiple classes in a single proxy install introduces a whole host of collision concerns that are entirely outside the spec. Merging Gateways into a single proxy instance makes understanding and correctly designing your configuration (and our implementation) much more complicated. Gateway merger is at least explicitly covered in the spec, but it's been an area where we've continued to find complicated edge cases with no good and intuitive behavior to codify.

Similarly merging GatewayClasses would add an additional layer to that complexity, and further isn't covered in the spec, so there's no roadmap for how we'd handle mutually incompatible configuration. Though there is broad language regarding this

Implementations MAY choose only one GatewayClass out of the pool of otherwise acceptable GatewayClasses if they can only reconcile one, or, if they are capable of reconciling multiple GatewayClasses, they may also choose as many as they like.

there's nothing guiding what reconciliation means or what conditions the implementation must satisfy. I take that to read more as that we could, for example, set up the operator to reconcile multiple classes by creating separate proxy instances for them.

mlavacca commented 11 months ago

Network restriction concerns aside, supporting multiple classes in a single proxy install introduces a whole host of collision concerns that are entirely outside the spec. Merging Gateways into a single proxy instance makes understanding and correctly designing your configuration (and our implementation) much more complicated. Gateway merger is at least explicitly covered in the spec, but it's been an area where we've continued to find complicated edge cases with no good and intuitive behavior to codify.

There is a WIP GEP related to merging gateways upstream (the first PR hasn't been merged yet). However, even with such a spec improvement, it won't be possible to merge gateways referring to different GatewayClasses, as in this issue example.

And a follow up question: is it possible for one single KIC deployment to be the implementation for multiple GatewayClasses?

Today It is possible to reconcile multiple GatewayClasses by using a single KIC instance (network concerns aside, as Travis mentioned), but they need to refer to the same ControllerName, as each KIC instance is bound to a single ControllerName:

Each implementation is expected to claim a domain-prefixed string value (like example.com/example-ingress) as its controllerName.

API Documentation states that one ControllerName must be picked by each implementation (even though the wording there is not that categorical, and it probably should be).

xiaoronglv commented 5 months ago

Hi @mlavacca, @mheap and @rainest ,

Good day!

From @czeslavo: You can set ingressController.env.gateway_api_controller_name to konghq.com/internet-controller) or konghq.com/private-controller to make KIC reconcile the GatewayClasses you've created.

I would like to follow this instruction and deploy two instances of Kong Gateway to my AWS EKS cluster. Here is my values.yaml for my second Kong instance, but it seems not to be working:

gateway:

  proxy:
    http:
      enabled: true
  autoscaling:
    enabled: true
  ingressController:
    env:
      gateway_api_controller_name: 'konghq.com/Kong-two-controller'

I used the following command to deploy the second instance:

helm install kong-2 kong/ingress -n kong -f ./values.yaml

Did you deploy your two Kong instances through Helm? How did you configure the Gateway Controller Name in the values.yaml file?

I couldn't find the definition either in the source code of ingress helm chart or kong helm chart.

I would appreciate it if you could share your values files as an example.

xiaoronglv commented 5 months ago

The kong document recommend to do it in this way, but it seems stales as well.

helm upgrade --install kong-internal kong/ingress -n internal --create-namespace --set controller.ingressController.ingressClass=internal
helm upgrade --install kong-external kong/ingress -n external --create-namespace --set controller.ingressController.ingressClass=external

Reference

sirTangale commented 1 week ago

The kong document recommend to do it in this way, but it seems stales as well.

helm upgrade --install kong-internal kong/ingress -n internal --create-namespace --set controller.ingressController.ingressClass=internal
helm upgrade --install kong-external kong/ingress -n external --create-namespace --set controller.ingressController.ingressClass=external

Reference

this if using kong ingress controller [ ingressclass api ] not gatewayapi [ new ]