Provide more realistic measurment conditions and more options for fine tuning experiments to improve the quality of experiments.
Deliverables
New experiment option to warm up targets and adapt request load
New experiment option to filter incoming requests
New metrics to monitor performance of a target compared to live
Why Are We Doing It?
The initial development of Thunderdome fixed several factors that affect experiments for simplicity. For example requests are filtered to only allow send based requests starting with /ipfs or /ipns. Also, while Thunderdome probes targets to establish whether they are ready to accept requests, it immediately sends the full request load making metrics gathered at the start of the experiment skewed since the target has no chance to fill caches or grow queues.
We want to provide more options for users to control some of these factors so that experiments can be more representative of real environments. We also want to refine the way experiments report metrics by adapting the delivered load to available capacity and giving the test targets a chance to warm up into a stable configuration which will produce higher quality and more accurate metrics.
What Is It?
Provide more realistic measurment conditions and more options for fine tuning experiments to improve the quality of experiments.
Deliverables
Why Are We Doing It?
The initial development of Thunderdome fixed several factors that affect experiments for simplicity. For example requests are filtered to only allow send based requests starting with /ipfs or /ipns. Also, while Thunderdome probes targets to establish whether they are ready to accept requests, it immediately sends the full request load making metrics gathered at the start of the experiment skewed since the target has no chance to fill caches or grow queues.
We want to provide more options for users to control some of these factors so that experiments can be more representative of real environments. We also want to refine the way experiments report metrics by adapting the delivered load to available capacity and giving the test targets a chance to warm up into a stable configuration which will produce higher quality and more accurate metrics.
Notes
Tasks