search

dotnet / BenchmarkDotNet

Powerful .NET library for benchmarking
https://benchmarkdotnet.org
MIT License
10.24k stars 952 forks source link

Cannot build NativeAOT 8.0 on Arch Linux #2548

Open Jisu-Woniu opened 3 months ago

Jisu-Woniu commented 3 months ago

I wrote a simple benchmark, with NativeAOT enabled:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;

BenchmarkRunner.Run<MyBench>();

[SimpleJob(RuntimeMoniker.NativeAot80)]
public class MyBench
{
    [Benchmark]
    public void Bench() { }
}

But when I run dotnet run -c Release, it fails with:

...
// Build Error: Standard output: 

 Standard error: 
 MSBuild version 17.8.5+b5265ef37 for .NET
$HOME/.nuget/packages/microsoft.net.illink.tasks/8.0.3/build/Microsoft.NET.ILLink.targets(210,5): warning : Property 'TrimmerDefaultAction' is deprecated in .NET 7 and higher and will be ignored. Use TrimMode instead. [<PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc/BenchmarkDotNet.Autogenerated.csproj]
  Generating native code
EXEC : error : Unrecognized instruction set avx-512f [<PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc/BenchmarkDotNet.Autogenerated.csproj]
  System.CommandLine.CommandLineException: Unrecognized instruction set avx-512f
     at System.CommandLine.Helpers.ConfigureInstructionSetSupport(String, Int32, Boolean, TargetArchitecture, TargetOS, String, String, Logger, Boolean) + 0x9d5
     at ILCompiler.Program.Run() + 0x59b
     at ILCompiler.ILCompilerRootCommand.<>c__DisplayClass227_0.<.ctor>b__0(ParseResult result) + 0x310
$HOME/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.3/build/Microsoft.NETCore.Native.targets(308,5): error MSB3073: The command ""$HOME/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/8.0.3/tools/ilc" @"obj/Release/net8.0/linux-x64/native/136c2e4d-c592-4318-baf9-67ab64a593fc.ilc.rsp"" exited with code 1. [<PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc/BenchmarkDotNet.Autogenerated.csproj]

// BenchmarkDotNet has failed to build the auto-generated boilerplate code.
// It can be found in <PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc
// Please follow the troubleshooting guide: https://benchmarkdotnet.org/articles/guides/troubleshooting.html
...

My dotnet --info output:

.NET SDK:
 Version:           8.0.103
 Commit:            6a90b4b4bc
 Workload version:  8.0.100-manifests.e99a2be4

Runtime Environment:
 OS Name:     arch
 OS Version:  
 OS Platform: Linux
 RID:         arch-x64
 Base Path:   /usr/share/dotnet/sdk/8.0.103/

.NET workloads installed:
 Workload version: 8.0.100-manifests.e99a2be4
There are no installed workloads to display.

Host:
  Version:      8.0.3
  Architecture: x64
  Commit:       9f4b1f5d66

.NET SDKs installed:
  8.0.103 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

It was installed from Arch Linux repository.

And my .csproj file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="BenchmarkDotNet" Version="0.13.12" />
  </ItemGroup>

</Project>
Jisu-Woniu commented 3 months ago

avx512f is supported on my CPU:

LANG=C lscpu | grep 512
Flags:   ... invpcid avx512f avx512dq rdseed adx smap avx512ifma ...

If I remove all avx-512* items from <PROJECT_DIR>/bin/Release/net8.0/<UUID>/BenchmarkDotNet.Autogenerated.csproj, it just compiles.

adamsitnik commented 3 months ago

Hello @Jisu-Woniu

Could you please go to <PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc/BenchmarkDotNet.Autogenerated.csproj and share with us what value was used for the <IlcInstructionSet>$this</IlcInstructionSet> element?

It should be just native and in theory should just work (https://github.com/dotnet/BenchmarkDotNet/pull/2464). I just need to check if it's BenchmarkDotNet or NativeAOT bug.

Jisu-Woniu commented 3 months ago

I build it again and this is the content of bin/Release/net8.0/f58553a8-0c73-499a-98cc-5ad358f5fe53/BenchmarkDotNet.Autogenerated.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <ImportDirectoryBuildProps>false</ImportDirectoryBuildProps>
    <ImportDirectoryBuildTargets>false</ImportDirectoryBuildTargets>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <RuntimeIdentifier>linux-x64</RuntimeIdentifier>
    <RuntimeFrameworkVersion></RuntimeFrameworkVersion>
    <AssemblyName>f58553a8-0c73-499a-98cc-5ad358f5fe53</AssemblyName>
    <AssemblyTitle>f58553a8-0c73-499a-98cc-5ad358f5fe53</AssemblyTitle>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <PlatformTarget>x64</PlatformTarget>
    <TreatWarningsAsErrors>False</TreatWarningsAsErrors>
    <DebugSymbols>false</DebugSymbols>
    <UseSharedCompilation>false</UseSharedCompilation>
    <Deterministic>true</Deterministic>
    <RunAnalyzers>false</RunAnalyzers>
    <PublishAot Condition=" '$(TargetFramework)' != 'net6.0' ">true</PublishAot>
    <IlcOptimizationPreference>Speed</IlcOptimizationPreference>
    <TrimMode>link</TrimMode><TrimmerDefaultAction>link</TrimmerDefaultAction>
    <IlcGenerateCompleteTypeMetadata>True</IlcGenerateCompleteTypeMetadata>
    <IlcGenerateStackTraceData>True</IlcGenerateStackTraceData>
    <EnsureNETCoreAppRuntime>false</EnsureNETCoreAppRuntime> <!-- workaround for 'This runtime may not be supported by.NET Core.' error -->
    <ErrorOnDuplicatePublishOutputFiles>false</ErrorOnDuplicatePublishOutputFiles> <!-- workaround for 'Found multiple publish output files with the same relative path.' error -->
    <ValidateExecutableReferencesMatchSelfContained>false</ValidateExecutableReferencesMatchSelfContained>
    <IlcInstructionSet>base,sse,sse2,sse3,ssse3,sse4.1,sse4.2,avx,avx2,avx-512f,avx-512bw,avx-512cd,avx-512dq,avx-512vbmi,aes,bmi,bmi2,fma,lzcnt,pclmul,popcnt</IlcInstructionSet>
  </PropertyGroup>
  <PropertyGroup>
<ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>true</ConcurrentGarbageCollection>
</PropertyGroup>

  <ItemGroup>
    <Compile Include="f58553a8-0c73-499a-98cc-5ad358f5fe53.notcs" Exclude="bin\**;obj\**;**\*.xproj;packages\**" />
  </ItemGroup>
  <ItemGroup>

    <ProjectReference Include="<PROJECT_DIR>/Bench.csproj" />
  </ItemGroup>
  <ItemGroup>
    <RdXmlFile Include="bdn_generated.rd.xml" />
  </ItemGroup>
</Project>

Maybe because the Instruction set should be avx512** instead of avx-512**?

Jisu-Woniu commented 3 months ago

I could set the IlcInstructionSet field to native using the following code:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;
using BenchmarkDotNet.Toolchains.NativeAot;

ManualConfig config = DefaultConfig.Instance.AddJob(
    Job.Default.WithToolchain(
        NativeAotToolchain.CreateBuilder().UseNuGet().IlcInstructionSet("native").ToToolchain()
    )
);

BenchmarkRunner.Run<MyBench>(config);

// Use custom runtime instead of the built-in one:
// [SimpleJob(RuntimeMoniker.NativeAot80)]
public class MyBench
{
    [Benchmark]
    public void Bench() { }
}

Generated .csproj file:


<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <ImportDirectoryBuildProps>false</ImportDirectoryBuildProps>
    <ImportDirectoryBuildTargets>false</ImportDirectoryBuildTargets>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <RuntimeIdentifier>linux-x64</RuntimeIdentifier>
    <RuntimeFrameworkVersion></RuntimeFrameworkVersion>
    <AssemblyName>6bb876b0-44dd-4195-9707-766e8b87018a</AssemblyName>
    <AssemblyTitle>6bb876b0-44dd-4195-9707-766e8b87018a</AssemblyTitle>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <PlatformTarget>x64</PlatformTarget>
    <TreatWarningsAsErrors>False</TreatWarningsAsErrors>
    <DebugSymbols>false</DebugSymbols>
    <UseSharedCompilation>false</UseSharedCompilation>
    <Deterministic>true</Deterministic>
    <RunAnalyzers>false</RunAnalyzers>
    <PublishAot Condition=" '$(TargetFramework)' != 'net6.0' ">true</PublishAot>
    <IlcOptimizationPreference>Speed</IlcOptimizationPreference>
    <TrimMode>link</TrimMode><TrimmerDefaultAction>link</TrimmerDefaultAction>
    <IlcGenerateCompleteTypeMetadata>True</IlcGenerateCompleteTypeMetadata>
    <IlcGenerateStackTraceData>True</IlcGenerateStackTraceData>
    <EnsureNETCoreAppRuntime>false</EnsureNETCoreAppRuntime> <!-- workaround for 'This runtime may not be supported by.NET Core.' error -->
    <ErrorOnDuplicatePublishOutputFiles>false</ErrorOnDuplicatePublishOutputFiles> <!-- workaround for 'Found multiple publish output files with the same relative path.' error -->
    <ValidateExecutableReferencesMatchSelfContained>false</ValidateExecutableReferencesMatchSelfContained>
    <IlcInstructionSet>native</IlcInstructionSet>
  </PropertyGroup>
  <PropertyGroup>
<ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>true</ConcurrentGarbageCollection>
</PropertyGroup>

  <ItemGroup>
    <Compile Include="6bb876b0-44dd-4195-9707-766e8b87018a.notcs" Exclude="bin\**;obj\**;**\*.xproj;packages\**" />
  </ItemGroup>
  <ItemGroup>

    <ProjectReference Include="<PROJECT_DIR>/Bench.csproj" />
  </ItemGroup>
  <ItemGroup>
    <RdXmlFile Include="bdn_generated.rd.xml" />
  </ItemGroup>
</Project>

It can compile and produce correct result.

adamsitnik commented 3 months ago

It can compile and produce correct result.

Big thanks for letting me know!

There must be some kind of bug here:

https://github.com/dotnet/BenchmarkDotNet/blob/4ab69be430b74469359d70d4557d2ed039e661ce/src/BenchmarkDotNet/Toolchains/NativeAot/Generator.cs#L231-L234

I'll try to fix it after the weekend, I am glad you are unblocked for now.

Jisu-Woniu commented 3 months ago

And maybe these should be replaced with avx512** too.

https://github.com/dotnet/BenchmarkDotNet/blob/4ab69be430b74469359d70d4557d2ed039e661ce/src%2FBenchmarkDotNet%2FToolchains%2FNativeAot%2FGenerator.cs#L251-L255

Treit commented 1 month ago

I am seeing this issue on Windows as well, does not seem to be Linux-specific.

eNeRGy164 commented 2 weeks ago

Same issue on my Windows PC after switching from a Intel Core I7 to an AMD Ryzen 5.

adamsitnik commented 4 days ago

Same issue on my Windows PC after switching from a Intel Core I7 to an AMD Ryzen 5.

@eNeRGy164 could you please attach the debugger and see what is happening here:

https://github.com/dotnet/BenchmarkDotNet/blob/4ab69be430b74469359d70d4557d2ed039e661ce/src/BenchmarkDotNet/Toolchains/NativeAot/Generator.cs#L231-L234

swtrse commented 18 hours ago

I have the same Problem on a minimal Project. Actually the GettingStarted Example. This is whats happening in the GetCurrentProcessInstructionSets-method. I did flag the Class with [SimpleJob(RuntimeMoniker.NativeAot80)] Platform: X64 TargetFrameworkMoniker: "net8.0" runtimeMoniker: Net80 If I manually set runtimeMoiniker to NativeAot80 the test seams to work.

My guess is the way 

if (!ConfigParser.TryParse(TargetFrameworkMoniker, out RuntimeMoniker runtimeMoniker))
{
    throw new NotSupportedException($"Invalid TFM: '{TargetFrameworkMoniker}'");
}

is written runtimeMoniker can never get a value other than what TargetFrameworkMoniker is givien. Since for NativeAot80 TargetFrameworkMoniker is net8.0 runtimeMoniker can only be Net80 and that is wrong since we want NativeAot80.

Also I do find the line runtimeMoniker >= RuntimeMoniker.NativeAot80 dangerous Is it realy given that ever entry in the enum now and in the funture that comes after NativeAot80 supports native? What happens if there is some similar requirement that needs some other sorting that invalidates the current sorting of the enum and is incompatible to the current statement?