Stringbuilder freezes the test...

[DBenS]

Hi.

I'm trying to benchmark two simple functions, where "Global_Var" strings with ~30 characters each.

    <Benchmark(Baseline:=True)>
    Public Function SB1() As String
        Public_StringBuilder.Clear()

        Public_StringBuilder.Append(Global_Var1)
        Public_StringBuilder.Append(Global_Var2)
        Public_StringBuilder.Append(Global_Var3)
        Public_StringBuilder.Append("apeas um teste de troca de valores XXX por YYY")

        Return Public_StringBuilder.ToString.Replace("XXX", "DESSE TYPO X").Replace("YYY", "por ESTE AQUI")
    End Function

    <Benchmark()>
    Public Function SB2() As String
        Public_StringBuilder.Clear()

        Public_StringBuilder.Append(Global_Var1)
        Public_StringBuilder.Append(Global_Var2)
        Public_StringBuilder.Append(Global_Var3)
        Public_StringBuilder.Append("apeas um teste de troca de valores XXX por YYY")

        Public_StringBuilder.Replace("XXX", "DESSE TYPO X")
        Public_StringBuilder.Replace("YYY", "por ESTE AQUI")

        Return Public_StringBuilder.ToString
    End Function

The problem is related to:

1- The test never ends, consuming 15%-22% of an i9-7900X w/64Gb RAM.

2- Even if I define "no WarmUp" and other settings, BDN performs them.

3- When I type CTRL-C on the CMD window, the job continues indefinitely, and I have to call Task Manager and kill it

See this print, similar to all others I got in 2 hours trying to conduct this test.

My Job parameters are:

ManualConfig.CreateEmpty().AddJob(Job.Default.WithRuntime(ClrRuntime.Net48) _
            .WithPlatform(Platform.AnyCpu) _
            .WithJit(Jit.RyuJit) _
            .WithWarmupCount(0) _
            .WithStrategy(RunStrategy.ColdStart) _
            .WithMaxIterationCount(1) _
            .WithMaxWarmupCount(0) _
            .WithLaunchCount(1) _
            .WithBaseline(True))

        Dim summary = BenchmarkRunner.Run(Of Benchmarks)()

Thanks for any help...

[timcassell]

Did it freeze? Or it just keep doing more iterations? Your results are extremely volatile which seems to be why it keeps running more iterations. It will do up to 100 iterations by default, and it looks like you only allowed it to run up to 36.

As for why it does warmup iterations, I'm not sure. Maybe try Job.Dry instead of Job.Default.

[DBenS]

I'm used to running DNB for three years, and I had never seen it:

• consuming so many interactions (the usual here is 13-20), and
• consuming 10x more time per interaction than other more complex tests.

It is just a single StringBuilder replacement with less than 100 chars string...

And whenever I stopped the test with CTRL-C, the executable remained running in its thread and consuming 15-25% of the CPU time.

PS: DryJob didn't solve the problem...

<DryJob>
<SimpleJob(RuntimeMoniker.Net48)>
<SimpleJob(Platform.AnyCpu)>
<SimpleJob(Jit.RyuJit)>

[MichalPetryka]

BenchmarkDotNet always makes a lot of iterations when the iterations vary in time by a noticeable amount, I've seen it do 100 iterations in some cases.