search

Kong / kong

🦍 The Cloud-Native API Gateway and AI Gateway.
https://konghq.com/install/#kong-community
Apache License 2.0
39.29k stars 4.82k forks source link

Kong via k8s - memory spikes at an hourly frequency #13859

Open himeshmyk opened 4 days ago

himeshmyk commented 4 days ago

Is there an existing issue for this?

Kong version ($ kong version)

2.4.1

Current Behavior

Hello,

Summary Experiencing memory spikes every hour with Kong 2.4.1 deployed with kubernetes arm64. No related to request count

Details We are currently using Kong in DB mode as an API Gateway in our organization. The version of Kong open source currently in use is 2.4.1.

Configuration Details We have deployed this via kubernetes with:

  1. arm64 architecture
  2. a replica count of 8 in one of the namespaces
  3. per pod (replica) memory and CPU configurations as defined below: resources: requests: memory: "1024M" cpu: "0.5" limits: memory: "2048M" cpu: "2"

Issue We're seeing pod memory spikes at an hourly frequency in each of the pods in the deployment: image

Some points to note:

  1. There is no significant difference in the number of requests during and outside the spike durations.
  2. This deployment is serving a peak request count of around 6.4k requests per second (~0.8k requests per second for each replica assuming correct load balancing).
  3. Further, the kong latency of requests also increases during these spikes, sometimes reaching ~2.5 seconds.

Ask As this is causing issues for our internal and external consumers, and we'd like to continue using Kong as an API gateway, with a further increase in usage over time, can you please share with us:

  1. Any leads on how to resolve this?
  2. Else, any pointers on further investigation for this.

Please let us know if you need any further information from our side.

Thanks, Himesh Bhatia

Expected Behavior

No response

Steps To Reproduce

No response

Anything else?

No response

xianghai2 commented 3 days ago

The recommendation is to use Kong 3.8 .