search

dotnet / BenchmarkDotNet

Powerful .NET library for benchmarking
https://benchmarkdotnet.org
MIT License
10.51k stars 968 forks source link

[FR] Support arbitrary process calls ala `["python", "-c", "8*9"]` #1893

Open NightMachinery opened 2 years ago

NightMachinery commented 2 years ago

I am wondering if adding support for benchmarking arbitrary process calls is possible for this project. Something like https://github.com/sharkdp/hyperfine/ , though perhaps with fewer shellisms.

adamsitnik commented 2 years ago

Hi @NightMachinary

Few years ago I've prototyped a proof of concept: https://github.com/dotnet/BenchmarkDotNet/pull/1050

I did that as I was hoping that the .NET Performance Team is going to be interested in using it. They have decided to implement their own solution, you can read more about it here: https://github.com/dotnet/performance/blob/5d6fc749333ce7d7e34e6d0ac311df67c5d2047c/docs/scenarios-workflow.md

Since there was no interest in the feature, I've closed the PR. In theory I could still resurrect it, but I am not sure if this would be a good time investment because as you have pointed out, there are already some open and cross-platform tools that can do that.

NightMachinery commented 2 years ago

Is BenchmarkDotNet Windows-only?

adamsitnik commented 2 years ago

Is BenchmarkDotNet Windows-only?

It's cross platform, works on every OS supported by .NET

NightMachinery commented 2 years ago

Then if you add this feature, this will be one of the very few opensource cross-platform tools to support it, and possibly the most sophisticated. (I have not found a single tool that benchmarks system commands with regards to both time and peak memory consumption.)