Multiple KIC instances in same namespace

tienhuynh17 commented 2 months ago

Hi, thanks for your great work on KIC. I have a problem when trying to install 2 KIC instances on the same namespace.

The first instance: I install by command:

helm upgrade --install kong1 kong/ingress -n kong --create-namespace --values ./values1.yaml

The values1.yaml files:

gateway:
  plugins:
   configMaps:
    - name: kong1-plugin-auth
      pluginName: kong1-plugin-auth
    - name: kong2-plugin-auth
      pluginName: kong2-plugin-auth
controller:
  ingressController:
    env:
      gateway_api_controller_name: konghq.com/kong1-gateway-controller

It works well.

The second instance: Then I install by command:

helm upgrade --install kong2 kong/ingress -n kong --create-namespace --values ./values2.yaml

The values2.yaml files:

gateway:
  plugins:
   configMaps:
    - name: kong1-plugin-auth
      pluginName: kong1-plugin-auth
    - name: kong2-plugin-auth
      pluginName: kong2-plugin-auth
controller:
  ingressController:
    env:
      gateway_api_controller_name: konghq.com/kong2-gateway-controller

Deployment/kong2-gateway logs the error below:

10.244.2.196 - - [21/Aug/2024:06:51:25 +0000] "GET /status HTTP/2.0" 200 1184 "-" "kong-ingress-controller/3.2.3"
2024/08/21 06:51:33 [notice] 1323#0: *5841 [lua] ready.lua:111: fn(): not ready for proxying: no configuration available (empty configuration present), client: 10.244.2.1, server: kong_status, request: "GET /status/ready HTTP/1.1", host: "10.244.2.198:8100"

Confimap kong1-plugin-auth, kong2-plugin-auth already created before. Does this error occur because 2 instances are running in the same namespace ? I researched and can not find it. Do I have to separate 2 instances in 2 different namespaces ?

Any help will be appreciated, Thank you.

joran-fonjallaz commented 1 month ago

hello, we have 2 kong KIC instances (green and blue) running in the same namespace without issues (if they have the same custom plugins)

they are configured as such:

the ingress controller config is (values.yaml file) is

ingress:
  controller:
    ingressController:
      ingressClass: kong-green
      # see https://docs.konghq.com/kubernetes-ingress-controller/latest/reference/cli-arguments/
      env:
        ingress_class: kong-green

similarly, for the blue instance, replace kong-green by kong-blue. You get the idea. Finally, those values must also be applied on the Gateway and GatewayClass

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: kong-green
  annotations:
    konghq.com/gatewayclass-unmanaged: 'true'
spec:
  controllerName: konghq.com/kong-green
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong-green
spec:
  gatewayClassName: kong-green

...

if you have different custom plugins in the kong instances, they will fail --> see https://github.com/Kong/charts/issues/1103. So make sure you are exactly the same custom plugins in both kong instances.

also, the ingressClass is repeated twice, as it also needs to be set in the env vars --> https://github.com/Kong/charts/issues/1100

good luck

tienhuynh17 commented 1 week ago

Many thanks @joran-fonjallaz, it works well for me. Here is my config (kong2):

gateway:
  plugins:
   configMaps:
    - name: kong1-plugin-auth
      pluginName: kong1-plugin-auth
    - name: kong2-plugin-auth
      pluginName: kong2-plugin-auth
controller:
  ingressController:
    ingressClass: kong2-gateway-controller
    env:
      gateway_api_controller_name: konghq.com/kong2-gateway-controller
      ingress_class: kong2-gateway-controller

Kong / charts

Multiple KIC instances in same namespace #1117