search

protocol / prodeng

Issues, discussions and documentation from the production engineering team
2 stars 1 forks source link

Create corpus of gateway requests #3

Closed iand closed 2 years ago

iand commented 2 years ago

Extract a set of requests that can be replayed against a gateway to evaluate TTFB performance. Ideally 30 days of successful requests, including duplicates over time so we can see the effects of any cache tuning we perform.

iand commented 2 years ago

Gathering via kibana but would be a lot faster with https://github.com/protocol/bifrost-infra/issues/1842

iand commented 2 years ago

Waiting on https://github.com/protocol/bifrost-infra/issues/1842 and https://github.com/protocol/bifrost-infra/pull/1850

iand commented 2 years ago

After some research I think this is not going to be practical or the best way to measure TTFB. One problem is building a corpus that may include links to undesirable content. We can't practically filter for all of those which means we must restrict access to the corpus. Another problem is that retrieval performance is highly dependent on transient network conditions that will vary hourly and daily. A static corpus can't accurately represent the state of the network at any future point in time.

A better approach would be to use a/b setup with mirrored traffic to compare metrics between versions of the Gateway