ElementsProject / lightning

Core Lightning — Lightning Network implementation focusing on spec compliance and performance
Other
2.81k stars 893 forks source link

add benchmark to phoronix-test-suite/openbenchmarking #4917

Open elkimek opened 2 years ago

elkimek commented 2 years ago

I'd love to bench c-lightning benchmark more often with larger scale of hardware (mostly CPUs) as that's my main job but last time I tried it, I ran into many issues and spent hours to get it running.

It would be best to add the benchmark to https://github.com/phoronix-test-suite/phoronix-test-suite and https://openbenchmarking.org/ so people can ran it with single command. That way we could get large dataset of hardware and of improvements being made across different versions.

Thanks

cdecker commented 2 years ago

Longtime Phoronix follower myself, sounds like an awesome idea. Question however is what aspect of the nodes do we want to benchmark? Options are throughput, latencies, number of channels, etc and some of them are hard to test in isolation, or aren't that decisive (throughput may be good, but synchronicity in the protocol may force us to be bottlenecked on latency).

Once we have decided on the benchmark dimensions we could instrument the nodes to make an integration possible 👍