search

dotnet / BenchmarkDotNet

Powerful .NET library for benchmarking
https://benchmarkdotnet.org
MIT License
10.49k stars 963 forks source link

How to benchmark x86 and x64 version of .NET Core using a single config? #856

Closed AlexeiScherbakov closed 3 years ago

AlexeiScherbakov commented 6 years ago

I use BenchmarkDotNet 0.11.0 for launching x86 & x64 .NET Core benchmarks (.NET Framework works fine) on Windows 10 x64. I looked at this - https://github.com/dotnet/BenchmarkDotNet/issues/534 and make config for benchmarks:

public class MultipleRuntimesConfig
        : ManualConfig
    {
        public MultipleRuntimesConfig()
        {
            // .NET
            if (Environment.Is64BitOperatingSystem)
            {
                Add(Job.RyuJitX64.With(CsProjClassicNetToolchain.Net47).WithId(".NET 4.7 x86_64"));
            }
            Add(Job.LegacyJitX86.With(CsProjClassicNetToolchain.Net47).WithId(".NET 4.7 x86"));

            // .NET Core
            if (Environment.Is64BitOperatingSystem)
            {
                var dotnetCli = NetCoreAppSettings
                    .NetCoreApp20
                    .WithCustomDotNetCliPath(@"C:\Program Files\dotnet\dotnet.exe", "64 bit cli");
                Add(Job.RyuJitX64.With(Platform.X64).With(CsProjCoreToolchain.From(dotnetCli)).WithId("64 bit cli"));

                var dotnetCli32bit = NetCoreAppSettings
                    .NetCoreApp20
                    .WithCustomDotNetCliPath(@"C:\Program Files (x86)\dotnet\dotnet.exe", "32 bit cli");
                Add(Job.RyuJitX86.With(Platform.X86).With(CsProjCoreToolchain.From(dotnetCli32bit)).WithId("32 bit cli"));
            }
            else
            {
                var dotnetCli = NetCoreAppSettings
                    .NetCoreApp20
                    .WithCustomDotNetCliPath(@"C:\Program Files\dotnet\dotnet.exe", "32 bit cli");
                Add(Job.RyuJitX86.With(Platform.X86).With(CsProjCoreToolchain.From(dotnetCli)).WithId("32 bit cli"));
            }

            Add(new ConsoleLogger());
            Add(DefaultExporters.Plain);
            Add(DefaultColumnProviders.Instance);

        }
    }

For starting benchmark I use

            ManualConfig config = null;
            //normal windows benchmark
            config = new MultipleRuntimesConfig();
            // raspberry benchmark
            //config = new RaspberyPiConfig();

            LoadLibraryHelper.Init();

            var summary = BenchmarkRunner.Run<MyBenchmark>(config);

Benchmark project file is:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFrameworks>net47;netcoreapp2.0</TargetFrameworks>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="BenchmarkDotNet" Version="0.11.0" />
        <PackageReference Include="System.Security.Cryptography.Cng" Version="4.5.0" />
    </ItemGroup>
</Project>

I have installed both .NET Core SDK (x86 & x64)

%MSBuildSdksPath% Env var is pointed to c:\Program Files\dotnet\sdk\2.1.400\Sdks\

But .NET Core x86 Benchmark cannot be started. Throwed exception An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B) when performing build Build Exception: c:\Program Files\dotnet\sdk\2.1.400\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.DefaultItems.targets(212,5): error MSB4062

What is wrong?

adamsitnik commented 6 years ago

Hi @AlexeiScherbakov

Most probably you just need to set PlatformTarget to AnyCPU for your project with benchmarks (AFAIK dotnet cli builds everything as x64 by default):

<PlatformTarget>AnyCPU</PlatformTarget>

If it still does not work you can try following config :

public class MultipleRuntimesConfig : ManualConfig
{
    public MultipleRuntimesConfig()
    {
        // .NET
        if (Environment.Is64BitOperatingSystem)
        {
            Add(Job.RyuJitX64.With(CsProjClassicNetToolchain.Net47).WithId(".NET 4.7 x86_64"));
        }
        Add(Job.LegacyJitX86.With(CsProjClassicNetToolchain.Net47).WithId(".NET 4.7 x86"));

        // .NET Core
        if (Environment.Is64BitOperatingSystem)
        {
            var dotnetCli = NetCoreAppSettings
                .NetCoreApp20
                .WithCustomDotNetCliPath(@"C:\Program Files\dotnet\dotnet.exe", "64 bit cli");
            Add(Job.Core.With(Platform.X64).With(CsProjCoreToolchain.From(dotnetCli)).WithId("64 bit cli"));

            var dotnetCli32bit = NetCoreAppSettings
                .NetCoreApp20
                .WithCustomDotNetCliPath(@"C:\Program Files (x86)\dotnet\dotnet.exe", "32 bit cli");
            Add(Job.Core.With(Platform.X86).With(CsProjCoreToolchain.From(dotnetCli32bit)).WithId("32 bit cli"));
        }
        else
        {
            Add(Job.Core.With(CsProjCoreToolchain.NetCoreApp20));
        }

        Add(new ConsoleLogger());
        Add(DefaultExporters.Plain);
        Add(DefaultColumnProviders.Instance);
    }
}

Please let me know if it helps

AlexeiScherbakov commented 6 years ago

It not helping, In both cases - benchmark trying to load task "CheckForImplicitPackageReferenceOverrides" from "c:\Program Files\dotnet\sdk\2.1.400\Sdks\Microsoft.NET.Sdk\targets..\tools\netcoreapp2.0/Microsoft.NET.Build.Tasks.dll and failed with 'Exception from HRESULT: 0x8007000B'

adamsitnik commented 3 years ago

I've just created a very similar app with the same dependencies and downloaded x86 and x64 dotnet SDK. 

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="BenchmarkDotNet" Version="0.12.1" />
    <PackageReference Include="System.Security.Cryptography.Cng" Version="4.7.0" />
  </ItemGroup>
</Project>
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;
using BenchmarkDotNet.Toolchains.CsProj;
using BenchmarkDotNet.Toolchains.DotNetCli;
using System.Security.Cryptography;

namespace ConsoleApp51
{
    class Program
    {
        static void Main(string[] args)
        {
            var x86core21 = Job.ShortRun
                .WithPlatform(BenchmarkDotNet.Environments.Platform.X86)
                .WithToolchain(CsProjCoreToolchain.From(NetCoreAppSettings.NetCoreApp21.WithCustomDotNetCliPath(@"C:\Projects\performance\tools\dotnet\x86\dotnet.exe")))
                .WithId("x86 .NET Core 2.1"); // displayed in the results table

            var x64core21 = Job.ShortRun
                .WithPlatform(BenchmarkDotNet.Environments.Platform.X64)
                .WithToolchain(CsProjCoreToolchain.From(NetCoreAppSettings.NetCoreApp21.WithCustomDotNetCliPath(@"C:\Projects\performance\tools\dotnet\x64\dotnet.exe")))
                .WithId("x64 .NET Core 2.1");

            var config = DefaultConfig.Instance
                .AddJob(x86core21)
                .AddJob(x64core21);

            BenchmarkRunner.Run<Perf_CryptoConfig>(config);
        }
    }

    public class Perf_CryptoConfig
    {
        [Benchmark]
        [Arguments("X509Chain")]
        public object CreateFromName(string name) => CryptoConfig.CreateFromName(name);
    }
}

Everything works as expected:

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18363.1139 (1909/November2018Update/19H2)
Intel Xeon CPU E5-1650 v4 3.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=5.0.100-rc.2.20479.15
  [Host]            : .NET Core 2.1.22 (CoreCLR 4.6.29220.03, CoreFX 4.6.29220.01), X64 RyuJIT
  x64 .NET Core 2.1 : .NET Core 2.1.11 (CoreCLR 4.6.27617.04, CoreFX 4.6.27617.02), X64 RyuJIT
  x86 .NET Core 2.1 : .NET Core 2.1.11 (CoreCLR 4.6.27617.04, CoreFX 4.6.27617.02), X86 RyuJIT

Toolchain=.NET Core 2.1  IterationCount=3  LaunchCount=1
WarmupCount=3

|         Method |               Job | Platform |      name |     Mean |     Error |    StdDev |
|--------------- |------------------ |--------- |---------- |---------:|----------:|----------:|
| CreateFromName | x64 .NET Core 2.1 |      X64 | X509Chain | 4.537 us | 0.8196 us | 0.0449 us |
| CreateFromName | x86 .NET Core 2.1 |      X86 | X509Chain | 3.426 us | 0.1016 us | 0.0056 us |

Full output:

```log PS C:\Users\adsitnik\source\repos\ConsoleApp51> dotnet run -c Release -f netcoreapp2.1 // Validating benchmarks: // ***** BenchmarkRunner: Start ***** // ***** Found 2 benchmark(s) in total ***** // ***** Building 2 exe(s) in Parallel: Start ***** // ***** Done, took 00:00:16 (16.89 sec) ***** // Found 2 benchmarks: // Perf_CryptoConfig.CreateFromName: x64 .NET Core 2.1(Platform=X64, Toolchain=.NET Core 2.1, IterationCount=3, LaunchCount=1, WarmupCount=3) [name=X509Chain] // Perf_CryptoConfig.CreateFromName: x86 .NET Core 2.1(Platform=X86, Toolchain=.NET Core 2.1, IterationCount=3, LaunchCount=1, WarmupCount=3) [name=X509Chain] // ************************** // Benchmark: Perf_CryptoConfig.CreateFromName: x64 .NET Core 2.1(Platform=X64, Toolchain=.NET Core 2.1, IterationCount=3, LaunchCount=1, WarmupCount=3) [name=X509Chain] // *** Execute *** // Launch: 1 / 1 // Execute: C:\Projects\performance\tools\dotnet\x64\dotnet.exe "53e51506-9f15-4496-b156-4e7e8d2b2ce0.dll" --benchmarkName "ConsoleApp51.Perf_CryptoConfig.CreateFromName(name: "X509Chain")" --job "x64 .NET Core 2.1" --benchmarkId 0 in C:\Users\adsitnik\source\repos\ConsoleApp51\bin\Release\netcoreapp2.1\53e51506-9f15-4496-b156-4e7e8d2b2ce0\bin\Release\netcoreapp2.1 // BeforeAnythingElse // Benchmark Process Environment Information: // Runtime=.NET Core 2.1.11 (CoreCLR 4.6.27617.04, CoreFX 4.6.27617.02), X64 RyuJIT // GC=Concurrent Workstation // Job: x64 .NET Core 2.1(Platform=X64, IterationCount=3, LaunchCount=1, WarmupCount=3) OverheadJitting 1: 1 op, 435300.00 ns, 435.3000 us/op WorkloadJitting 1: 1 op, 3022800.00 ns, 3.0228 ms/op OverheadJitting 2: 16 op, 812400.00 ns, 50.7750 us/op WorkloadJitting 2: 16 op, 892900.00 ns, 55.8063 us/op WorkloadPilot 1: 16 op, 99400.00 ns, 6.2125 us/op WorkloadPilot 2: 32 op, 169500.00 ns, 5.2969 us/op WorkloadPilot 3: 64 op, 337000.00 ns, 5.2656 us/op WorkloadPilot 4: 128 op, 774300.00 ns, 6.0492 us/op WorkloadPilot 5: 256 op, 1242100.00 ns, 4.8520 us/op WorkloadPilot 6: 512 op, 2438900.00 ns, 4.7635 us/op WorkloadPilot 7: 1024 op, 4858200.00 ns, 4.7443 us/op WorkloadPilot 8: 2048 op, 9494600.00 ns, 4.6360 us/op WorkloadPilot 9: 4096 op, 18793000.00 ns, 4.5881 us/op WorkloadPilot 10: 8192 op, 37820200.00 ns, 4.6167 us/op WorkloadPilot 11: 16384 op, 76598600.00 ns, 4.6752 us/op WorkloadPilot 12: 32768 op, 149292700.00 ns, 4.5561 us/op WorkloadPilot 13: 65536 op, 294846000.00 ns, 4.4990 us/op WorkloadPilot 14: 131072 op, 597533600.00 ns, 4.5588 us/op OverheadWarmup 1: 131072 op, 468300.00 ns, 3.5728 ns/op OverheadWarmup 2: 131072 op, 464900.00 ns, 3.5469 ns/op OverheadWarmup 3: 131072 op, 460600.00 ns, 3.5141 ns/op OverheadWarmup 4: 131072 op, 465000.00 ns, 3.5477 ns/op OverheadWarmup 5: 131072 op, 464900.00 ns, 3.5469 ns/op OverheadWarmup 6: 131072 op, 460600.00 ns, 3.5141 ns/op OverheadWarmup 7: 131072 op, 460600.00 ns, 3.5141 ns/op OverheadWarmup 8: 131072 op, 461900.00 ns, 3.5240 ns/op OverheadActual 1: 131072 op, 466600.00 ns, 3.5599 ns/op OverheadActual 2: 131072 op, 462400.00 ns, 3.5278 ns/op OverheadActual 3: 131072 op, 462100.00 ns, 3.5255 ns/op OverheadActual 4: 131072 op, 465800.00 ns, 3.5538 ns/op OverheadActual 5: 131072 op, 464400.00 ns, 3.5431 ns/op OverheadActual 6: 131072 op, 463200.00 ns, 3.5339 ns/op OverheadActual 7: 131072 op, 466500.00 ns, 3.5591 ns/op OverheadActual 8: 131072 op, 462000.00 ns, 3.5248 ns/op OverheadActual 9: 131072 op, 464900.00 ns, 3.5469 ns/op OverheadActual 10: 131072 op, 460600.00 ns, 3.5141 ns/op OverheadActual 11: 131072 op, 460600.00 ns, 3.5141 ns/op OverheadActual 12: 131072 op, 460600.00 ns, 3.5141 ns/op OverheadActual 13: 131072 op, 460600.00 ns, 3.5141 ns/op OverheadActual 14: 131072 op, 458800.00 ns, 3.5004 ns/op OverheadActual 15: 131072 op, 462000.00 ns, 3.5248 ns/op WorkloadWarmup 1: 131072 op, 605068100.00 ns, 4.6163 us/op WorkloadWarmup 2: 131072 op, 597548400.00 ns, 4.5589 us/op WorkloadWarmup 3: 131072 op, 600938200.00 ns, 4.5848 us/op // BeforeActualRun WorkloadActual 1: 131072 op, 601893500.00 ns, 4.5921 us/op WorkloadActual 2: 131072 op, 591062200.00 ns, 4.5094 us/op WorkloadActual 3: 131072 op, 592473100.00 ns, 4.5202 us/op // AfterActualRun WorkloadResult 1: 131072 op, 601431400.00 ns, 4.5886 us/op WorkloadResult 2: 131072 op, 590600100.00 ns, 4.5059 us/op WorkloadResult 3: 131072 op, 592011000.00 ns, 4.5167 us/op // AfterAll // Benchmark Process 39892 has exited with code 0 Mean = 4.537 us, StdErr = 0.026 us (0.57%), N = 3, StdDev = 0.045 us Min = 4.506 us, Q1 = 4.511 us, Median = 4.517 us, Q3 = 4.553 us, Max = 4.589 us IQR = 0.041 us, LowerFence = 4.449 us, UpperFence = 4.615 us ConfidenceInterval = [3.717 us; 5.357 us] (CI 99.9%), Margin = 0.820 us (18.07% of Mean) Skewness = 0.36, Kurtosis = 0.67, MValue = 2 // ************************** // Benchmark: Perf_CryptoConfig.CreateFromName: x86 .NET Core 2.1(Platform=X86, Toolchain=.NET Core 2.1, IterationCount=3, LaunchCount=1, WarmupCount=3) [name=X509Chain] // *** Execute *** // Launch: 1 / 1 // Execute: C:\Projects\performance\tools\dotnet\x86\dotnet.exe "dc1c038e-3e78-48b0-97ea-6720a395020d.dll" --benchmarkName "ConsoleApp51.Perf_CryptoConfig.CreateFromName(name: "X509Chain")" --job "x86 .NET Core 2.1" --benchmarkId 0 in C:\Users\adsitnik\source\repos\ConsoleApp51\bin\Release\netcoreapp2.1\dc1c038e-3e78-48b0-97ea-6720a395020d\bin\Release\netcoreapp2.1 // BeforeAnythingElse // Benchmark Process Environment Information: // Runtime=.NET Core 2.1.11 (CoreCLR 4.6.27617.04, CoreFX 4.6.27617.02), X86 RyuJIT // GC=Concurrent Workstation // Job: x86 .NET Core 2.1(Platform=X86, IterationCount=3, LaunchCount=1, WarmupCount=3) OverheadJitting 1: 1 op, 404500.00 ns, 404.5000 us/op WorkloadJitting 1: 1 op, 2930200.00 ns, 2.9302 ms/op OverheadJitting 2: 16 op, 731300.00 ns, 45.7063 us/op WorkloadJitting 2: 16 op, 823100.00 ns, 51.4438 us/op WorkloadPilot 1: 16 op, 83700.00 ns, 5.2313 us/op WorkloadPilot 2: 32 op, 149300.00 ns, 4.6656 us/op WorkloadPilot 3: 64 op, 254900.00 ns, 3.9828 us/op WorkloadPilot 4: 128 op, 477600.00 ns, 3.7313 us/op WorkloadPilot 5: 256 op, 1099200.00 ns, 4.2938 us/op WorkloadPilot 6: 512 op, 1865100.00 ns, 3.6428 us/op WorkloadPilot 7: 1024 op, 3626700.00 ns, 3.5417 us/op WorkloadPilot 8: 2048 op, 7236200.00 ns, 3.5333 us/op WorkloadPilot 9: 4096 op, 14611400.00 ns, 3.5672 us/op WorkloadPilot 10: 8192 op, 29769400.00 ns, 3.6340 us/op WorkloadPilot 11: 16384 op, 58720900.00 ns, 3.5840 us/op WorkloadPilot 12: 32768 op, 117619700.00 ns, 3.5895 us/op WorkloadPilot 13: 65536 op, 228678900.00 ns, 3.4894 us/op WorkloadPilot 14: 131072 op, 453691000.00 ns, 3.4614 us/op WorkloadPilot 15: 262144 op, 911532100.00 ns, 3.4772 us/op OverheadWarmup 1: 262144 op, 836900.00 ns, 3.1925 ns/op OverheadWarmup 2: 262144 op, 825900.00 ns, 3.1506 ns/op OverheadWarmup 3: 262144 op, 840100.00 ns, 3.2047 ns/op OverheadWarmup 4: 262144 op, 842800.00 ns, 3.2150 ns/op OverheadWarmup 5: 262144 op, 827200.00 ns, 3.1555 ns/op OverheadWarmup 6: 262144 op, 827300.00 ns, 3.1559 ns/op OverheadWarmup 7: 262144 op, 828000.00 ns, 3.1586 ns/op OverheadWarmup 8: 262144 op, 831500.00 ns, 3.1719 ns/op OverheadWarmup 9: 262144 op, 827800.00 ns, 3.1578 ns/op OverheadActual 1: 262144 op, 831200.00 ns, 3.1708 ns/op OverheadActual 2: 262144 op, 827200.00 ns, 3.1555 ns/op OverheadActual 3: 262144 op, 953800.00 ns, 3.6385 ns/op OverheadActual 4: 262144 op, 828000.00 ns, 3.1586 ns/op OverheadActual 5: 262144 op, 825900.00 ns, 3.1506 ns/op OverheadActual 6: 262144 op, 826000.00 ns, 3.1509 ns/op OverheadActual 7: 262144 op, 827200.00 ns, 3.1555 ns/op OverheadActual 8: 262144 op, 840100.00 ns, 3.2047 ns/op OverheadActual 9: 262144 op, 830100.00 ns, 3.1666 ns/op OverheadActual 10: 262144 op, 831500.00 ns, 3.1719 ns/op OverheadActual 11: 262144 op, 840100.00 ns, 3.2047 ns/op OverheadActual 12: 262144 op, 824100.00 ns, 3.1437 ns/op OverheadActual 13: 262144 op, 831400.00 ns, 3.1715 ns/op OverheadActual 14: 262144 op, 1028600.00 ns, 3.9238 ns/op OverheadActual 15: 262144 op, 824500.00 ns, 3.1452 ns/op WorkloadWarmup 1: 262144 op, 917746900.00 ns, 3.5009 us/op WorkloadWarmup 2: 262144 op, 898933600.00 ns, 3.4292 us/op WorkloadWarmup 3: 262144 op, 902640000.00 ns, 3.4433 us/op // BeforeActualRun WorkloadActual 1: 262144 op, 897754400.00 ns, 3.4247 us/op WorkloadActual 2: 262144 op, 898314200.00 ns, 3.4268 us/op WorkloadActual 3: 262144 op, 900516100.00 ns, 3.4352 us/op // AfterActualRun WorkloadResult 1: 262144 op, 896924300.00 ns, 3.4215 us/op WorkloadResult 2: 262144 op, 897484100.00 ns, 3.4236 us/op WorkloadResult 3: 262144 op, 899686000.00 ns, 3.4320 us/op // AfterAll // Benchmark Process 24896 has exited with code 0 Mean = 3.426 us, StdErr = 0.003 us (0.09%), N = 3, StdDev = 0.006 us Min = 3.421 us, Q1 = 3.423 us, Median = 3.424 us, Q3 = 3.428 us, Max = 3.432 us IQR = 0.005 us, LowerFence = 3.415 us, UpperFence = 3.436 us ConfidenceInterval = [3.324 us; 3.527 us] (CI 99.9%), Margin = 0.102 us (2.97% of Mean) Skewness = 0.32, Kurtosis = 0.67, MValue = 2 // ***** BenchmarkRunner: Finish ***** // * Export * BenchmarkDotNet.Artifacts\results\ConsoleApp51.Perf_CryptoConfig-report.csv BenchmarkDotNet.Artifacts\results\ConsoleApp51.Perf_CryptoConfig-report-github.md BenchmarkDotNet.Artifacts\results\ConsoleApp51.Perf_CryptoConfig-report.html // * Detailed results * Perf_CryptoConfig.CreateFromName: x64 .NET Core 2.1(Platform=X64, Toolchain=.NET Core 2.1, IterationCount=3, LaunchCount=1, WarmupCount=3) [name=X509Chain] Runtime = .NET Core 2.1.11 (CoreCLR 4.6.27617.04, CoreFX 4.6.27617.02), X64 RyuJIT; GC = Concurrent Workstation Mean = 4.537 us, StdErr = 0.026 us (0.57%), N = 3, StdDev = 0.045 us Min = 4.506 us, Q1 = 4.511 us, Median = 4.517 us, Q3 = 4.553 us, Max = 4.589 us IQR = 0.041 us, LowerFence = 4.449 us, UpperFence = 4.615 us ConfidenceInterval = [3.717 us; 5.357 us] (CI 99.9%), Margin = 0.820 us (18.07% of Mean) Skewness = 0.36, Kurtosis = 0.67, MValue = 2 -------------------- Histogram -------------------- [4.465 us ; 4.629 us) | @@@ --------------------------------------------------- Perf_CryptoConfig.CreateFromName: x86 .NET Core 2.1(Platform=X86, Toolchain=.NET Core 2.1, IterationCount=3, LaunchCount=1, WarmupCount=3) [name=X509Chain] Runtime = .NET Core 2.1.11 (CoreCLR 4.6.27617.04, CoreFX 4.6.27617.02), X86 RyuJIT; GC = Concurrent Workstation Mean = 3.426 us, StdErr = 0.003 us (0.09%), N = 3, StdDev = 0.006 us Min = 3.421 us, Q1 = 3.423 us, Median = 3.424 us, Q3 = 3.428 us, Max = 3.432 us IQR = 0.005 us, LowerFence = 3.415 us, UpperFence = 3.436 us ConfidenceInterval = [3.324 us; 3.527 us] (CI 99.9%), Margin = 0.102 us (2.97% of Mean) Skewness = 0.32, Kurtosis = 0.67, MValue = 2 -------------------- Histogram -------------------- [3.416 us ; 3.437 us) | @@@ --------------------------------------------------- // * Summary * BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18363.1139 (1909/November2018Update/19H2) Intel Xeon CPU E5-1650 v4 3.60GHz, 1 CPU, 12 logical and 6 physical cores .NET Core SDK=5.0.100-rc.2.20479.15 [Host] : .NET Core 2.1.22 (CoreCLR 4.6.29220.03, CoreFX 4.6.29220.01), X64 RyuJIT x64 .NET Core 2.1 : .NET Core 2.1.11 (CoreCLR 4.6.27617.04, CoreFX 4.6.27617.02), X64 RyuJIT x86 .NET Core 2.1 : .NET Core 2.1.11 (CoreCLR 4.6.27617.04, CoreFX 4.6.27617.02), X86 RyuJIT Toolchain=.NET Core 2.1 IterationCount=3 LaunchCount=1 WarmupCount=3 | Method | Job | Platform | name | Mean | Error | StdDev | |--------------- |------------------ |--------- |---------- |---------:|----------:|----------:| | CreateFromName | x64 .NET Core 2.1 | X64 | X509Chain | 4.537 us | 0.8196 us | 0.0449 us | | CreateFromName | x86 .NET Core 2.1 | X86 | X509Chain | 3.426 us | 0.1016 us | 0.0056 us | // * Legends * name : Value of the 'name' parameter Mean : Arithmetic mean of all measurements Error : Half of 99.9% confidence interval StdDev : Standard deviation of all measurements 1 us : 1 Microsecond (0.000001 sec) // ***** BenchmarkRunner: End ***** // ** Remained 0 benchmark(s) to run ** Run time: 00:00:13 (13.47 sec), executed benchmarks: 2 Global total time: 00:00:30 (30.38 sec), executed benchmarks: 2 // * Artifacts cleanup * ```

FWIW if anyone wanted to extend the config with Full .NET Framework, it should look like this:

var x86core21 = Job.ShortRun
    .WithPlatform(BenchmarkDotNet.Environments.Platform.X86)
    .WithToolchain(CsProjCoreToolchain.From(NetCoreAppSettings.NetCoreApp21.WithCustomDotNetCliPath(@"C:\Projects\performance\tools\dotnet\x86\dotnet.exe")))
    .WithId("x86 .NET Core 2.1"); // displayed in the results table

var x64core21 = Job.ShortRun
    .WithPlatform(BenchmarkDotNet.Environments.Platform.X64)
    .WithToolchain(CsProjCoreToolchain.From(NetCoreAppSettings.NetCoreApp21.WithCustomDotNetCliPath(@"C:\Projects\performance\tools\dotnet\x64\dotnet.exe")))
    .WithId("x64 .NET Core 2.1");

var x86net461 = Job.ShortRun
    .WithPlatform(BenchmarkDotNet.Environments.Platform.X86)
    .WithToolchain(CsProjClassicNetToolchain.Net461)
    .WithId("x86 .NET 4.6.1"); // displayed in the results table

var x64net461 = Job.ShortRun
    .WithPlatform(BenchmarkDotNet.Environments.Platform.X64)
    .WithToolchain(CsProjClassicNetToolchain.Net461)
    .WithId("x64 .NET 4.6.1");

var config = DefaultConfig.Instance
    .AddJob(x86core21)
    .AddJob(x64core21)
    .AddJob(x86net461)
    .AddJob(x64net461);

Since I was not able to reproduce the problem I am closing the issue.