petabridge / NBench

Performance benchmarking and testing framework for .NET applications :chart_with_upwards_trend:
https://nbench.io/
Apache License 2.0
533 stars 46 forks source link

dotnet-nbench can accidentally try to execute [assembly-name].deps.json instead of the assembly #259

Closed Aaronontheweb closed 4 years ago

Aaronontheweb commented 5 years ago

Happens on nodes running .NET Core 2.2 and later.

shawndube commented 5 years ago

Is there a workaround/alternative for now?

shawndube commented 5 years ago

I tried changing the project target but realized the issue was really with the CLI. I got it working by: 1) changing the version of the CLI for that project by adding a global.json (ref: https://markheath.net/post/switching-between-netcore-sdk-versions )

2) i then had an error about not having 2.1 for a framework that required adding this to the csproj file in the PropertyGroup:

2.1.6

3) Then had error about packages restored with different runtime than build/publish. Fix for that was just to run "dotnet restore" from the cli again.

After that, i was able to run "dotnet nbench" against that project fine.

Aaronontheweb commented 5 years ago

Thanks for posting the detailed workaround! @izavala and I are looking at doing another big usability overhaul for NBench 2.0 and this will be on our list for that.

anttirt commented 5 years ago

Just in case someone runs into this, but like me can't fiddle with the target frameworks or global.json, another work-around is to use the undocumented --fx-version {version} flag with dotnet nbench, for example:

fxversion=$(dotnet --list-runtimes | \
    grep Microsoft.NETCore.App | \
    awk '{ print $2 }' | \
    tail -1)

dotnet nbench --fx-version $fxversion

Or:

$fxversion = dotnet --list-runtimes | `
    select-string "Microsoft.NETCore.App" | `
    select-object -last 1 | `
    foreach-object { $data = $_ -split " "; $data[1] }

dotnet nbench --fx-version $fxversion

The reason the command fails without this is that the runner formats a dotnet exec command with the fxVersion variable, and when it's empty, this results in --fx-version --depsfile "Foo.deps.json". This in turn is parsed by dotnet exec as if --depsfile were the value for the --fx-version flag, and the following Foo.deps.json is interpreted as the assembly file to run.

The offending line: https://github.com/petabridge/NBench/blob/557f2fbca250a4a45636f5e4b41b58b8440b33f2/src/NBench.Runner.DotNetCli/Program.cs#L284

This patch should fix it but I don't have time to do proper testing and/or a pull request right now:

diff --git a/src/NBench.Runner.DotNetCli/Program.cs b/src/NBench.Runner.DotNetCli/Program.cs
index c45b32e..417319c 100644
--- a/src/NBench.Runner.DotNetCli/Program.cs
+++ b/src/NBench.Runner.DotNetCli/Program.cs
@@ -281,7 +281,10 @@ namespace NBench.Runner.DotNetCli
             var depsFile = targetFileNameWithoutExtension + ".deps.json";
             var runtimeConfigJson = targetFileNameWithoutExtension + ".runtimeconfig.json";

-            var args = $@"exec --fx-version {fxVersion} --depsfile ""{depsFile}"" ";
+            var args = $@"exec --depsfile ""{depsFile}"" ";
+
+            if (!string.IsNullOrWhiteSpace(fxVersion))
+                args += $"--fx-version {fxVersion} ";

             if (File.Exists(Path.Combine(workingDirectory, runtimeConfigJson)))
                 args += $@"--runtimeconfig ""{runtimeConfigJson}"" ";
Aaronontheweb commented 4 years ago

Resolved via NBench 2.0.0 https://github.com/petabridge/NBench/releases/tag/2.0.0 - no more dotnet nbench.