search

dotnet / BenchmarkDotNet

Powerful .NET library for benchmarking
https://benchmarkdotnet.org
MIT License
10.57k stars 970 forks source link

Benchmarks work in .NET 4.7.2 but not .NET Core 3.1 #1433

Closed peschkaj closed 4 years ago

peschkaj commented 4 years ago

When running a benchmark with .NET Framework 4.7.2, benchmarks proceed as expected. However, switching to .NET Core 3.1 results in a host of errors stating Generate Exception: Unable to find <Project> in <Folder> and its subfolders.

Running git diff shows the only difference being in the .csproj` file:

diff --git a/Source/Tests/PerformanceTests/PerformanceTests.csproj b/Source/Tests/PerformanceTests/PerformanceTests.csproj
index 793405c703..6a69534058 100644
--- a/Source/Tests/PerformanceTests/PerformanceTests.csproj
+++ b/Source/Tests/PerformanceTests/PerformanceTests.csproj
@@ -3,7 +3,7 @@
     <RootNamespace>RedGate.SqlMonitor.PerformanceTests</RootNamespace>
     <AssemblyName>RedGate.SqlMonitor.PerformanceTests</AssemblyName>
     <ImplicitlyExpandNETStandardFacades>False</ImplicitlyExpandNETStandardFacades>
-    <TargetFramework>net472</TargetFramework>
+    <TargetFramework>netcoreapp3.1</TargetFramework>
   </PropertyGroup>
   <ItemGroup>
     <PackageReference Include="BenchmarkDotNet" Version="0.12.1" />
peschkaj commented 4 years ago

Renaming PerformanceTests.csproj to have a name that matches the assembly name fixes this error, however a new error crops up: error MSB4086: A numeric comparison was attempted on "$(LangVersion)" that evaluates to "latest" instead of a number, in condition "'$(LangVersion)' == '' Or '$(LangVersion)' < '7.3'".

update manually setting the version number fixes this. I'm not sure what the appropriate behavior should be in this scenario.

adamsitnik commented 4 years ago

Generate Exception: Unable to find in and its subfolders

This is expected. To build the auto-generated .dll we need to reference the project that defines the benchmarks and we search for something like typeof(TypeWithBenchmarks).Assembly.Name + ".csproj". If the names dont match, we simply can't find it (afaik we print a nice error with explanation).

A numeric comparison was attempted on

This is a bug that I've introduced in 0.12.1. It was already fixed in https://github.com/dotnet/BenchmarkDotNet/pull/1420 but we have not released 0.12.2 with a fix yet.

manually setting the version number fixes this

Great, thanks for sharing the workaround. I think that you can either keep using it until we ship 0.12.2 or downgrade to 0.12.0.

Thanks, Adam

peschkaj commented 4 years ago

That all makes sense and will be a good incentive to clean up assembly names and project file names.

Thanks for clearing the language version issues, as well. I think our workaround makes sense for now, our benchmarking needs are limited at the moment.