dotnet / BenchmarkDotNet

Powerful .NET library for benchmarking
https://benchmarkdotnet.org
MIT License
10.47k stars 962 forks source link

Autogenerated project is not able to reference a self-contained executable #2076

Open ShinyCraig opened 2 years ago

ShinyCraig commented 2 years ago

I'm trying to benchmark code in a self-contained project and am encountering an issue where the autogenerated project is not being generated as a self-contained project leading to this error when I try to run my benchmark.

// Build Error: Standard output: 

 Standard error: 
 Time Elapsed 00:00:00.41
    1 Error(s)
    0 Warning(s)
C:\Program Files\dotnet\sdk\6.0.303\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(1096,5): error NETSDK1151: The referenced project '..\..\..\..\..\..\ExeProject\ExeProject.csproj' is a self-contained executable.  A self-contained
executable cannot be referenced by a non self-contained executable.  For more information, see https://aka.ms/netsdk1151 [C:\Branches\Benchmarknet6win\Benchmarknet6win\bin\Release\net6.0-windows\win-x64\583881de-f153-4b54-9427-92e7509799da\Be
nchmarkDotNet.Autogenerated.csproj]
Build FAILED.
C:\Program Files\dotnet\sdk\6.0.303\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(1096,5): error NETSDK1151: The referenced project '..\..\..\..\..\..\ExeProject\ExeProject.csproj' is a self-contained executable.  A self-contained
executable cannot be referenced by a non self-contained executable.  For more information, see https://aka.ms/netsdk1151 [C:\Branches\Benchmarknet6win\Benchmarknet6win\bin\Release\net6.0-windows\win-x64\583881de-f153-4b54-9427-92e7509799da\Be
nchmarkDotNet.Autogenerated.csproj]
Copyright (C) Microsoft Corporation. All rights reserved.
Microsoft (R) Build Engine version 17.2.0+41abc5629 for .NET

// BenchmarkDotNet has failed to build the auto-generated boilerplate code.
// It can be found in C:\Branches\Benchmarknet6win\Benchmarknet6win\bin\Release\net6.0-windows\win-x64\583881de-f153-4b54-9427-92e7509799da
// Please follow the troubleshooting guide: https://benchmarkdotnet.org/articles/guides/troubleshooting.html

All of the projects in my benchmarking solution have true in them, but the generated csproj does not. I'm unsure how to properly set up my projects such that the project generated by benchmarkdotnet is also SelfContained, or if there's another way to do this without making my original code not self-contained I am all ears.

The set up of my project is I have a exe project with some code I want to test. A benchmark project with some code that calls code from the exe project, and a "runner" project that calls var result = BenchmarkRunner.Run<MyBenchmark>();

Thanks for any assistance!

adamsitnik commented 2 years ago

Hi @ShinyCraig

Self contained apps work well with InProcessToolchain (#1685).

You can also try passing an MSBuild argument to BDN:

BenchmarkSwitcher
   .FromAssembly(typeof(Program).Assembly)
   .Run(args,
       DefaultConfig.Instance
            .AddJob(Job.Default.WithArguments(new Argument[] { new MsBuildArgument("/p:SelfContained=true") })));

but I am not sure if it's going to work, as the default toolchain builds the app instead of publishing it.

ShinyCraig commented 2 years ago

Hi @adamsitnik

Thanks for the help! That got me closer to running things, but I'm running in circles now.

If I use something very similar to what you've provided me, I still get an error.

// Build Error: Standard output: 

 Standard error: 
 Time Elapsed 00:00:00.22
    1 Error(s)
    0 Warning(s)
C:\Program Files\dotnet\sdk\6.0.303\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(154,5): error NETSDK1031: It is not supported to build or publish a self-contained application without specifying a RuntimeIdentifier. You must either specify a RuntimeIdentifier or set SelfContai
ned to false. [C:\......\bin\Release\net6.0-windows\win-x64\e40eb31a-8564-48e0-a2d0-d125e9d44dfe\BenchmarkDotNet.Autogenerated.csproj]

So I try specifying a RuntimeIdentifier, by either doing

Job.Default.WithArguments(new Argument[]
{
    new MsBuildArgument("/p:SelfContained=true"),
    new MsBuildArgument("/p:RuntimeIdentifier=win-x64")
});

or

Job.Default.WithArguments(new Argument[]
{
    new MsBuildArgument("/p:SelfContained=true,RuntimeIdentifier=win-x64")
});

Either way it goes back to the original error...

// Build Error: Standard output: 

 Standard error: 
 Time Elapsed 00:00:00.47
    1 Error(s)
    0 Warning(s)
C:\Program Files\dotnet\sdk\6.0.303\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(1096,5): error NETSDK1151: The referenced project '..\..\..\..\..\..\CognitiveToolkit.Cli\CognitiveToolkit.Cli.csproj' is a self-contained executable.  A self-contained executable cannot be referenced by a non self-cont
ained executable.  For more information, see https://aka.ms/netsdk1151 [C:\......\bin\Release\net6.0-windows\win-x64\647eb3f5-8d17-485c-bc44-9bdd462f305e\BenchmarkDotNet.Autogenerated.csproj]
Build FAILED.

Apologies if this is a stupid question, but am I simply missing the proper way to pass both arguments in?

Thanks for any help you can provide, it's greatly appreciated.

adamsitnik commented 2 years ago

@ShinyCraig is there any chance you could provide small repro?

ShinyCraig commented 2 years ago

Hi @adamsitnik

Apologies for the delay, here's a repo that demonstrates the issue I've been having.

https://github.com/ShinyCraig/benchmarkissuedemo

program.cs in BenchmarkRunner is where you'll want to look, I tried to comment and provide as much context as I can, but let me know if you need any other information.

ShinyCraig commented 2 years ago

Hi @adamsitnik , any updates?

Thanks!

timcassell commented 1 year ago

This is fixed by #2393.

timcassell commented 1 year ago

Actually, it looks like this was fixed in 0.13.2. Can you try with a more recent version (latest is 0.13.7) @ShinyCraig?