Open Stebalien opened 5 years ago
cc @ipfs/wg-infrastructure
As part of the end-to-end benchmarking, can we look at some tooling around the storage of the benchmarks, over just comparing on a PR by PR basis, i.e. https://github.com/golang/perf or https://github.com/influxdata/grade
@lanzafame I've added a item to the todo list.
@Stebalien I have updated the issue with the point about RC telemetry.
On the issue of Network Simulation for libp2p TestLab, I've come across this: https://chepeftw.github.io/NS3DockerEmulator/ - which is a docker emulator that can then be imported in NS-3 (https://nsnam.org). The production IPFS/libp2p/DHT code is packaged in a docker container, which represents a node in the network. Through NS-3 we then do the network setup and run the simulation (it's more emulation than simulation TBH). This can produce realistic results using production code. It might not be useful for all purposes, but surely it can be for many cases.
I don't think Dial Latency (testing the dialer, transports, etc.).
is an IPFS part, I think this test is libp2p releated because its impossible to fix any problem about that in IPFS without changing the libp2p codebase, also that test and his result impact all libp2p users, such as filecoin, IPFS, ethereum 2.0...
@Jorropo these are tests needed by go-ipfs, not necessarily tests of go-ipfs. That is, we want to test these libp2p features before we cut a release.
ipfs p2p
).In both network tests, we should track and compare metrics on:
The Plan: https://docs.google.com/spreadsheets/d/1xyqyGUF-oe3x9ln88YonVeOMWWdknik74lVgL_3dBY8