search

omgnetwork / omg-childchain-v2

pronounced /Ch-ch/
Apache License 2.0
5 stars 2 forks source link

interpreting results of performance tests #149

Closed ayrat555 closed 3 years ago

ayrat555 commented 4 years ago

related issues/prs:

https://github.com/omgnetwork/childchain/issues/59 https://github.com/omgnetwork/childchain/issues/141 https://github.com/omgnetwork/childchain/issues/145 https://github.com/omgnetwork/elixir-omg/pull/1745

During code review discussions, the question about the interpretation of performance tests raised several times.

Currently, by default successful outcome is defined by error_rate mean = 0. Optionally you can check error_rate percentile (10%-90%) = 0.

boolafish commented 4 years ago

I think for several tests it will need:

  1. server side API latency (each API should have one metric). P9x version instead of max.
  2. client side error rate
  3. server side error rate (optional?)

Personally I think this is test specific. So it might be better to allow each test to inject each logic that it want to check and developers can add new checks for each test. (eg. the API for deposit test would be different from a watcher one)

boolafish commented 4 years ago

https://omgnetworkhq.slack.com/archives/CMX790Q5V/p1603252908142800?thread_ts=1603249243.137000&channel=CMX790Q5V&message_ts=1603252908.142800

Add feedback from infra team

ayrat555 commented 4 years ago

Ok. I'll try adding metrics like: https://app.datadoghq.com/dashboard/rvw-u8w-hh3/traefik-ingress-metrics?from_ts=1603462350880&live=true&to_ts=1603465950880

https://www.weave.works/blog/the-red-method-key-metrics-for-microservices-architecture/

InoMurko commented 3 years ago

I think this is all done