izgeri
commented
3 years ago I think we can potentially put this card off until we have some initial data. If we've defined how we'll collect the data, what experiments we'll run, and how we'll aggregate the data from the experiments - then we can define the report structure once we've collected some data and have a better understanding of how to visualize it. Do you agree, or is it important to do this now?
Overview
Assume there is a pipeline to measure, export and analyse metrics. This issue is about defining the potential formats of benchmarking reports. For example, for streaming latency we might generate a report providing measures of spread, distributions etc.
Definition of done