Closed AbhigyaWangoo closed 3 hours ago
Checked and verified that we were suffering from overloaded CPU utilizations with AWS cloudwatch EC2 metric CPUUtilization
. Checking for any relevant deployments...
Looked at all deployments on October 20th, seeing one that introduced potentially compute intensive code: https://github.com/CirroePlatform/Hermes/commit/334023fa55d83d558688816c82f2eaf4eb26382e
Engaging oncall to rollback. Rollback command should be git reset --hard HEAD~1
LGTM, running command and closing out
During peak hours, our latencies spike up 2-3x the normal amount. We only started to notice this issue around October 20th, and it has been consistently happening since then. Can someone help