bbc / sqs-consumer

Build Amazon Simple Queue Service (SQS) based applications without the boilerplate
https://bbc.github.io/sqs-consumer/
Other
1.74k stars 333 forks source link

[Bug]: High CPU usage #415

Closed dungvv-qsoft closed 1 year ago

dungvv-qsoft commented 1 year ago

Describe the bug

- Issue: High cpu usage

Testing on the local server to be able to generate flame graph

With Queue enabled, It shows the initial spike is too high on resources occupied image

This is non queue test, the initial spike is not too high compared with queue turning on: image

Evidence: .cpuprofile and .text report files for 2 cases (turn on/off queue when do stress test) https://github.com/dungvv-qsoft/cpu-profile

Testing on our production server

(CPU/Ram usage graph - From 3:20 - 3:30) Maximum average CPU usage is almost 100% image

(CPU/Ram usage graph - From 4:15 to 4:30) Maximum average CPU usage is around 60% image

Your minimal, reproducible example

None

Steps to reproduce

Try to run a stress test on the api that using sqs-consumer lib -> The CPU usage increase significantly (It's not that high if we turn off the queue)

Expected behavior

CPU usage should be not high compared when turn off queue (with the same traffic)

How often does this bug happen?

Every time

Screenshots or Videos

No response

Platform

Package version

v5.7.0

AWS SDK version

v2.1106.0

Additional context

No response

nicholasgriffintn commented 1 year ago

I'm going to convert this to a discussion as without a proper production, there's no task for us here.

For a bug ticket, we'd need exactly the area that SQS Consumer is actually responsible and there are a few other variables at play here including how you are using SQS Consumer, what your infrastructure setup is, the aws-sdk library, etc, etc.

Feel free to raise an issue if you ever pin point the issue and it is with SQS Consumer.