gomodule / redigo

Go client for Redis
Apache License 2.0
9.76k stars 1.25k forks source link

Increase tail latencies when migrated service to arm64 architecture #651

Closed aman432 closed 1 year ago

aman432 commented 1 year ago

Ask questions at https://stackoverflow.com/questions/ask?tags=go+redis

Our service uses redigo as redis client library. Our application sees an rps of 4k. So for p99 requests the latencies are under the sla but for tail latencies we saw redis Get calls were taking more that 1second. We checked server for slow queries but there were no slow queries since all calls wew just GET calls. We also saw cpu usage and memory usage less than 50%. Number of go routines were also normal. The only anomaly was that when we migrated our application from aws ec2 to graviton then after few days tail latencies started increasing. Once we reverted the deployments from arm64 to amd64 instances the latencies became normal.

Can anybody help us understanding the root cause?

stevenh commented 1 year ago

Issues are for bugs in the module. Please use the link above to ask questions regarding usage.

This sounds like a hardware / virtualisation performance issue