IterationSetup allocates 400B

[AleksLigeza]

Hello! I would like to report that using [IterationSetup] is causing 400B to be allocated. I quickly checked how it works for earlier versions and it seems that the problem started in 0.13.6 and persists in 0.13.12.

Please check the summary and code below:

BenchmarkDotNet v0.13.12, Windows 10 (10.0.19045.4529/22H2/2022Update) 11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores .NET SDK 8.0.106 [Host] : .NET 8.0.6 (8.0.624.26715), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI Job-HVPPIJ : .NET 8.0.6 (8.0.624.26715), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

InvocationCount=1 UnrollFactor=1

Method	Mean	Error	Allocated
TestBenchmark	0.0 ns	0.0 ns	400 B

using BenchmarkDotNet.Attributes;

BenchmarkDotNet.Running.BenchmarkRunner.Run<Test>();

[MemoryDiagnoser]
public class Test
{
    [IterationSetup]
    public void TestSetup()
    {
    }

    [Benchmark]
    public void TestBenchmark()
    {
    }
}

If I remove IterationSetup or change it to GlobalSetup, the problem does not occur.

BenchmarkDotNet v0.13.12, Windows 10 (10.0.19045.4529/22H2/2022Update) 11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores .NET SDK 8.0.106 [Host] : .NET 8.0.6 (8.0.624.26715), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI DefaultJob : .NET 8.0.6 (8.0.624.26715), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

Method	Mean	Error	StdDev	Allocated
TestBenchmark	0.0698 ns	0.0279 ns	0.0466 ns	-

Thanks!

[timcassell]

This is because [IterationSetup] causes the benchmark method to be invoked only once. Removing it allows it to be invoked multiple times. There are background allocations that we cannot control (see https://github.com/dotnet/runtime/issues/101536), and more benchmark invocations drowns those out.

2562 will improve it somewhat, but will not completely eliminate the issue. The runtime will need to remove the background allocations to completely fix it, and that's out of our control.