search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.47k stars 4.76k forks source link

[API Proposal]: `RuntimeHelpers.IsMethodFullyOptimized` for benchmarking tools #99870

Open MichalPetryka opened 8 months ago

MichalPetryka commented 8 months ago

Background and motivation

Benchmarking tools like Benchmark.Net have to wait until methods get to Tier 1 to get reliable results today. Currently they do so by either calling the methods a few times and waiting an arbitrary amount of time which is not very reliable or by using runtime diagnostic events which makes writing benchmarks much harder. As such, exposing an API that lets users poll for methods getting to T1 (or code running in AOT or with tiering disabled) could make benchmarking of code noticeably easier.

API Proposal

namespace System.Runtime.CompilerServices;

public class RuntimeHelpers
{
    public static bool IsMethodFullyOptimized(RuntimeMethodHandle method);
}

API Usage

Action a = M;
while (!RuntimeHelpers.IsMethodFullyOptimized(a.Method.MethodHandle))
    a();

// run benchmarks

Alternative Designs

This could block the thread instead or be an event.

Risks

No response

EgorBo commented 8 months ago

This won't help BDN much, IMO. While benchmark method itself can be already optimized, it doesn't mean that all of its callees are. I don't think the problem you're trying to solve can be solved like that.

Maybe BDN can just parse ETW/ES events, check stacks, etc.