BenchmarkDotNet requires dotnet cli toolchain to be installed

[ViktorHofer]

I'm manually compiling my benchmark with corerun & csc like this (.NET Core 2.1 master):

C:\git\corefx\bin\runtime\netcoreapp-Windows_NT-Release-x64>.\CoreRun ..\..\..\tools\csc.dll /noconfig /optimize /r:System.Private.Corelib.dll /r:System.Runtime.dll /r:System.Runtime.Extensions.dll /r:System.Console.dll /r:System.Text.RegularExpressions.dll /r:System.Collections.dll /r:BenchmarkDotNet.dll /r:BenchmarkDotNet.Core.dll /out:regex-match.dll regex-match.cs

When running it with corerun I get the following error:

C:\git\corefx\bin\runtime\netcoreapp-Windows_NT-Release-x64>.\CoreRun regex-match.dll
BenchmarkDotNet requires dotnet cli toolchain to be installed, benchmark 'RegexBenchmark.RegexMatch: .NET Core 2.1 master(Runtime=Core, Toolchain=OutOfProcessToolchain)' will not be executed

I would expect this error if I wouldn't have provided a custom cli path WithCustomDotNetCliPath. This is my config:

// Job #1
            Add(Job.Default
                .With(Runtime.Core)
                .With(CsProjCoreToolchain.From(NetCoreAppSettings
                    .NetCoreApp21
                    .WithCustomDotNetCliPath(@"C:\dotnet\dotnet.exe", "OutOfProcessToolchain")))
                .WithId(".NET Core 2.1 master"));

            // Job #2
            Add(Job.Default
                .With(InProcessToolchain.Instance)
                 .With(Runtime.Core)
                 .WithId(".NET Core 2.1 regex"));

The InProc job runs as expected (Job 2) but the one which should you use a custom dotnet path doesn't work. Is this expected?

[adamsitnik]

@ViktorHofer it's not expected, it's a bug ;) Fixed

[ViktorHofer]

Now I get the following error:

// Generate Exception: Unable to find regex-perf in C:\git\corefx\bin\runtime\netcoreapp-Windows_NT-Release-x64 and its subfolders. Most probably the name of output exe is different than the name of the .(c/f)sproj

My compiled output is regex-perf.dll, it seems it looks for an exe?

[adamsitnik]

We search for .csproj file to reference it. When you build with csc there is no csproj.

But don't worry tomorrow I should have new toolchain for building against local CoreCLR+CoreFX without any need to use csc or CoreRun