search

buger / goreplay

GoReplay is an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data. It can be used to increase confidence in code deployments, configuration changes and infrastructure changes.
https://goreplay.org
Other
18.62k stars 21 forks source link

Statistics and analytics #772

Open slimus opened 4 years ago

slimus commented 4 years ago

We need to look some statistics and analytics while we replay traffic

How should it work? We should count how many requests\responses per sec goreplay get, how many request per path and per method goreplay get. If it is possible we should show avg (or median) of latency.

Which params we should show?

How should it look like?

Questions?

  1. Should we show any information from middleware?

@buger @arijitAD @urbanishimwe please review this issue and check my vision.

Go stats packages:

  1. https://github.com/segmentio/stats

  2. https://prometheus.io/docs/guides/go-application/

urbanishimwe commented 4 years ago

Latency

Do you mean latency for a certain request in case output-http-track-response is enabled?

slimus commented 4 years ago

@urbanishimwe it is good question. I think we should show different statistics when we are using different input\output. What do you think?

buger commented 4 years ago

In terms of architecture, I believe that each of the inputs and outputs should provide own analytics. So analytics API/UI in future will depend on inputs/outputs you have.

HTTP output contributes to a number of replayed requests, latencies (ideally with their distribution), requests per smth and etc. input raw can contribute data about how much requests it intercepts, what is the packet loss, the average size of the request and etc.

And so on.

slimus commented 4 years ago

Comments after sync:

  1. Outputstdout should parse of data to count requests and responses correct
  2. Check how many data and time prometheus exporter store the data.
  3. Check histogram type in prometheus exporter. How it works.
  4. Think about how to send statistic from slaves of goreplay to master, to collect it in one place.
  5. Read about latencies, for example how to chrome dev calculate it (or other browser with dev tools)