buger / goreplay

GoReplay is an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data. It can be used to increase confidence in code deployments, configuration changes and infrastructure changes.
https://goreplay.org
Other
18.42k stars 4 forks source link

Latency Metrics for simple static node server are all over the place. #997

Open lionsroar96 opened 2 years ago

lionsroar96 commented 2 years ago

Hey guys,

We have a very simple static node app that we are hosting on AWS ECS with launch type being Fargate. The issue we are facing is that the latency metrics are all over the place for the same static endpoint (that just responds with a 200 and a small body).

We're seeing latencies(converting from ns to ms) in the range of:

  1. 280542ms / 413310ms
  2. 612ms / 1730ms
  3. 5465ms / 3880ms

All for the same static endpoint. What could be causing this large variation? Its a very simple node app right now.

buger commented 2 years ago

Hi!

What is the expected latency of your app? How do you measure the latency, by looking to recorded files?

lionsroar96 commented 2 years ago

Thanks for the quick response. The latency we're seeing in prod is <1000ms correlating closer to the second output. We're replaying traffic twice to get these latency differences (running recorded traffic through a main container and a branch PR). Both are through "--output-http-track-response" flag.