--disasm switch on ARM64 throws Exception

kunalspathak commented 4 years ago

I get following exception when using --disasm on ARM64.

// AfterAll
Unhandled exception. System.InvalidOperationException: There is an error in XML document (0, 0).
 ---> System.Xml.XmlException: Root element is missing.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at System.Xml.XmlReader.MoveToContent()
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderDisassemblyResult.Read9_DisassemblyResult()
   --- End of inner exception stack trace ---
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   at BenchmarkDotNet.Diagnosers.WindowsDisassembler.Disassemble(DiagnoserActionParameters parameters)
   at BenchmarkDotNet.Diagnosers.DisassemblyDiagnoser.Handle(HostSignal signal, DiagnoserActionParameters parameters)
   at BenchmarkDotNet.Diagnosers.CompositeDiagnoser.Handle(HostSignal signal, DiagnoserActionParameters parameters)
   at BenchmarkDotNet.Loggers.SynchronousProcessOutputLoggerWithDiagnoser.ProcessInput()
   at BenchmarkDotNet.Toolchains.DotNetCli.DotNetCliExecutor.Execute(BenchmarkCase benchmarkCase, BenchmarkId benchmarkId, ILogger logger, ArtifactsPaths artifactsPaths, IDiagnoser diagnoser, String executableName, IResolver resolver)
   at BenchmarkDotNet.Toolchains.DotNetCli.DotNetCliExecutor.Execute(ExecuteParameters executeParameters)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.Execute(ILogger logger, BenchmarkCase benchmarkCase, BenchmarkId benchmarkId, IToolchain toolchain, BuildResult buildResult, IResolver resolver)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.RunCore(BenchmarkCase benchmarkCase, BenchmarkId benchmarkId, ILogger logger, IResolver resolver, BuildResult buildResult)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.Run(BenchmarkRunInfo benchmarkRunInfo, Dictionary`2 buildResults, IResolver resolver, ILogger logger, List`1 artifactsToCleanup, String resultsFolderPath, String logFilePath, StartedClock& runChronometer)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.Run(BenchmarkRunInfo[] benchmarkRunInfos)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config)
   at MicroBenchmarks.Program.Main(String[] args) in D:\Users\dotnet-bot\kpathak\git\performance\src\benchmarks\micro\Program.cs:line 35

kunalspathak commented 4 years ago

@adamsitnik

adamsitnik commented 4 years ago

The disassembler does not work on ARM & ARM64 due to the fact that we are internally using Iced which does not support it yet.

As soon as https://github.com/0xd4d/iced/issues/79 and https://github.com/0xd4d/iced/issues/80 get implemented, we are going to update Iced on our side and add the support. I don't know when it's going to happen.

adamsitnik commented 2 years ago

Currently we can implement it in following ways:

DOTNET_JitDisasm

Use DOTNET_JitDisasm exposed recently by https://github.com/dotnet/runtime/pull/73365 and parse the disassembly from standard output. In #2092 I've extended ExecuteResult with new StandardOutput that contains all the lines printed by the benchmark process to standard output.

https://github.com/dotnet/BenchmarkDotNet/blob/58f2d18d67e8d21b80c9ffef8301d3d924fd3e9f/src/BenchmarkDotNet/Toolchains/Results/ExecuteResult.cs#L22

Advantages:

if we add any new architectures in the future, it should work OOTB
no new dependencies
some users are already used to this format of output

Disadvantages:

it works only for method that are JITTed, R2R method are not included
works only with .NET 7+
printing to standard output will be visible in the profiles if the benchmark is being profiled at the same time
filtering by method names might require us to implement some kind of mapping (https://devblogs.microsoft.com/dotnet/performance_improvements_in_net_7/#jit)

capstone

The current x64/x86 disassembler uses ClrMD to get disassembly as byte[] and Iced to decode the instructions and format them. Iced does not support arm yet: https://github.com/icedland/iced/issues/72 (the issue is closed, but there is no support).

For arm64/arm we could use https://github.com/capstone-engine/capstone which supports .NET and arm. The .NET library has recently added support for linux and macOS (so far it supported only Windows): https://github.com/9ee1/Capstone.NET/pull/32, but a new NuGet package version was not published to nuget.org yet. However, a fork of it has published the package: https://www.nuget.org/packages/js6pak.Gee.External.Capstone

Advantages:

should work with all .NET versions

Disadvantages:

new native dependencies
afaik no support for arm64 on Windows (it's a matter of extending https://github.com/9ee1/Capstone.NET/pull/32)
the library is not very popular, we might hit some bugs and need to fix them

Iced

We could contribute to Iced and implement arm64 support. This would require us to establish new abstractions (so far everything assumed x64) and also implement arm64/arm decoding. IMO this would be a lot of work, but I might be wrong.

adamsitnik commented 2 years ago

@janvorli I've done two experiments:

I've tried ClrMD on Arm64: it works on Linux, it throws on Windows
I've tried Capstone and it works nice on Linux. I was able to decode all methods (using --disasmFilter *) and it did not throw.

Branch that I've been using: https://github.com/dotnet/BenchmarkDotNet/tree/arm64Disasm

Demo:

; BenchmarkDotNet.Samples.IntroDisassembly.SumField()
       stp x29, x30, [sp, #-0x10]!
       mov x29, sp
       mov w1, wzr
       mov w2, wzr
       ldr x0, [x0, #8]
       ldr w3, [x0, #8]
       cmp w3, #0
       b.le #0x1044
       mov x3, x0
       ldr w4, [x3, #8]
       cmp w2, w4
       b.hs #0x1040
       sxtw x4, w2
       lsl x4, x4, #2
       add x4, x4, #0x10
       ldr w3, [x3, x4]
       add w1, w3, w1
       add w2, w2, #1
       ldr w3, [x0, #8]
       cmp w3, w2
       b.gt #0xfd8
       mov w0, w1
       ldp x29, x30, [sp], #0x10
       ret
       bl #0xffffffffff8a39b0
       brk #0
; Total bytes of code 104

dotnet / BenchmarkDotNet