AndyAyersMS
commented
8 years ago Wondering now why we can't just invoke the xunit entry point. Seems like this ought to work. So basically Main emulates the (single iteration) activity of xunit-perf.
At any rate this is a bit more pressing now. I have some hacked up instrumentation in the jit that can capture method entry counts, and I can use this plus inline replay plus some differential math to deduce the call counts of particular call sites. So in principle I can compute the per-call perf impact of a particular inline.
However this is only reliable if both the entry count and instructions retired measurements come from consistent runtime executions. So if we measure one when running as an exe and another when running under xunit-perf, we may draw inaccurate conclusions, if the number of calls varies between the two execution methods.
We run a given benchmark both ways. There's a common subset of methods that get executed, but also sets unique to each run method. It would better if one execution method would hit all the relevant methods. Ideally this would be the exe path.
We likely can't fully invoke the xunit entry points (marked with
[benchmark]attributes) when running as an exe, but if there was some way to call them -- even if they had some "xunit not initialized" opt-out handling -- it would probably work out ok.