Open smudge202 opened 8 years ago
So I had a look at the MS Performance tools. I've not got as far as actually utilising them yet because the packages are not (from what I can see) published to NuGet.
I've raised an associated issue here:
I contemplated temporarily naughtily hosting the packages on an appveyor account feed if the packages worked and adding the respective NuGet.config
to the engine so that collabs on this project could be given permission to access them. However, @pharring has pointed out the packages already exists on a public myget feed (https://github.com/Microsoft/xunit-performance/issues/100#issuecomment-154520619).
Good thing, because I'm not sure appveyor have read-only permissions available on account feeds yet (https://github.com/appveyor/ci/issues/35).
I'll continue checking out the MS benchmarking as the MS fanboy I can be, but there are alternatives available: https://github.com/PerfDotNet/BenchmarkDotNet
My write-up on BenchmarkDotNet
:
https://github.com/PerfDotNet/BenchmarkDotNet/issues/33#issuecomment-154549833
No clue what's wrong; I'm going to assume it's trying to pull information out of the app domain in such a way that it doesn't work through DNX.
As for the MS variant, the current standard xunit runner doesn't detect the tests, and we can't add the dnx runner MS ships because it requires dnxcore50
, which for one reason or another is unavailable from dnx46
which the test project targets.
Not going well.
One or more of our issues (such as #7) explicitly request we perform some benchmarking so we can track the performance of components. So as not to hold up development on the other components, I'm transferring those requirements to this issue.
My intent is to create a Command Line utility that can spool up a dedicated azure VM which can perform stress / performance testing in order to ascertain benchmarks. I'll create the necessary directories in this repository so that changes to component systems and associated benchmarks can be stored alongside the changes.I'm probably going to use Compose to do this, so I'll drive that package towards completion of the 1.0.0-beta this week. Hopefully that way only benchmarking will be blocked by Compose.We need to evaluate and potentially implement one of the following two technologies (open to additional options if anyone knows of one):