Add support for alpine on arm64

andrewlock commented 2 months ago

Summary of changes

Adds support for the tracer on alpine on ARM64

Reason for change

We currently support x64 on glibc/alpine but only glibc on arm64. This closes that matrix which makes it easier to understand

Implementation details

There's several layers, but at a high level

Anywhere where we detect platform, make sure to include linux-musl-arm64 as well as linux-arm64
Build the tracer + profiler on alpine arm64

There were several difficulties

The ruby-based fpm bundler we use doesn't work on alpine arm64, hence #5905
I couldn't get our existing alpine.build.dockerfile to compile on arm64 (where we currently compile llvm/clang16 on alpine14).
- To work around it, created a separate alpine.build.arm64.dockerfile image which uses alpine:18 (the first version that includes clang16) as the base.
- Obviously this gives us different requirements in arm64, but I think that's fine because a) this is a new feature b) we already have differences on the glibc requirements for arm64 c) We only support .NET 5+ anyway on arm64 (.NET 6 really given .NET 5 is mostly gone) and they ship alpine:18 images for .NET 6+
- I created a multi arch base image from the two base images (as some bloke with a blog describes here)
I had difficulty compiling the profiler on arm64 initially (we require it for some crash tracking functionality), as we were getting undefined reference to_Uaarch64_get_accessors_int'` when trying to compile libunwind. @gleocadie tracked it down to this issue, and fixed it by just not building the libunwind tests 🙇‍♂️
The sqlite IAST instrumentation tests on alpine arm64 were failing in .NET 5. I think that's because the sqlite package we're using there doesn't support it (which makes sense) so just disabled the tests for that scenario. They pass in .NET 6+ so I'm not worried about it.
Removed the "noop" smoke tests stage (there's nothing to run in it currently)
We can't even install .NET Core 2.1 using the dotnet install script on alpine arm64, so bailing out. We can probably bail out of more too, but that's not necessary so haven't done it yet.

There's still one outstanding issue I can't get my head around, on debian x64 (i.e. not something that should have changed) The OnEolFrameworkInSsi_WhenForwarderPathExists_CallsForwarderWithExpectedTelemetry tests is failing by crashing. It's bizarre, I can't figure it out, if anyone has any ideas, please let me know 😅

Test coverage

I've run full installer + full TFM tests here 🤞

Other details

Stacked on

as they both change key parts of the build + will mean we need to rebuild the VMs

Fixes #3850

datadog-ddstaging[bot] commented 2 months ago

Datadog Report

Branch report: andrew/arm64-musl-support Commit report: a51bfd5 Test service: dd-trace-dotnet

:x: 1 Failed (1 Known Flaky), 358513 Passed, 2343 Skipped, 16h 31m 54.52s Total Time

:x: Failed Tests (1)

CheckApplicationDoesNotEndUpInDeadlock - Datadog.Profiler.IntegrationTests.LinuxOnly.DlIteratePhdrDeadlock - :snowflake: Known flaky - Details

Expand for error
``` The test Samples.Computer01 is running for too long or was lost ```

andrewlock commented 2 months ago

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

andrewlock commented 2 months ago

Benchmarks Report for appsec :snail:

Benchmarks for #5933 compared to master:

1 benchmarks are faster, with geometric mean 1.210
2 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster :tada: Same allocations :heavy_check_mark: #### Faster :tada: in #5933 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net6.0 | 1.210 | 173.28 | 143.16 |

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`AllCycleSimpleBody`|net6.0|74μs|128ns|496ns|0.0744| 0|0|6 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`AllCycleSimpleBody`|netcoreapp3.1|61.8μs|87.3ns|338ns|0.0617| 0|0|6.95 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`AllCycleSimpleBody`|net472|49.1μs|60.6ns|235ns|1.31| 0|0|8.34 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`AllCycleMoreComplexBody`|net6.0|79.7μs|104ns|389ns|0.12| 0|0|9.51 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`AllCycleMoreComplexBody`|netcoreapp3.1|68.8μs|79.4ns|297ns|0.137| 0|0|10.36 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`AllCycleMoreComplexBody`|net472|56.4μs|47.8ns|179ns|1.86| 0.0282|0|11.85 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`ObjectExtractorSimpleBody`|net6.0|173ns|0.127ns|0.49ns|0.00397| 0|0|280 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`ObjectExtractorSimpleBody`|netcoreapp3.1|219ns|0.342ns|1.32ns|0.00373| 0|0|272 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`ObjectExtractorSimpleBody`|net472|167ns|0.173ns|0.671ns|0.0446| 0|0|281 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`ObjectExtractorMoreComplexBody`|net6.0|3.05μs|0.915ns|3.42ns|0.053| 0|0|3.78 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`ObjectExtractorMoreComplexBody`|netcoreapp3.1|3.99μs|2.51ns|9.38ns|0.0499| 0|0|3.69 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`ObjectExtractorMoreComplexBody`|net472|3.79μs|2.6ns|9.73ns|0.603| 0.00568|0|3.8 KB| |#5933|`AllCycleSimpleBody`|net6.0|73.4μs|74.9ns|290ns|0.0725| 0|0|6.01 KB| |#5933|`AllCycleSimpleBody`|netcoreapp3.1|62.9μs|65.8ns|246ns|0.0944| 0|0|6.95 KB| |#5933|`AllCycleSimpleBody`|net472|49.6μs|51.1ns|198ns|1.31| 0|0|8.34 KB| |#5933|`AllCycleMoreComplexBody`|net6.0|79.9μs|81.9ns|306ns|0.119| 0|0|9.51 KB| |#5933|`AllCycleMoreComplexBody`|netcoreapp3.1|69.9μs|121ns|468ns|0.139| 0|0|10.37 KB| |#5933|`AllCycleMoreComplexBody`|net472|57.1μs|19.3ns|72.4ns|1.88| 0.0285|0|11.85 KB| |#5933|`ObjectExtractorSimpleBody`|net6.0|143ns|0.182ns|0.705ns|0.00396| 0|0|280 B| |#5933|`ObjectExtractorSimpleBody`|netcoreapp3.1|203ns|0.255ns|0.953ns|0.0037| 0|0|272 B| |#5933|`ObjectExtractorSimpleBody`|net472|170ns|0.186ns|0.721ns|0.0446| 0|0|281 B| |#5933|`ObjectExtractorMoreComplexBody`|net6.0|3.2μs|2.32ns|8.69ns|0.053| 0|0|3.78 KB| |#5933|`ObjectExtractorMoreComplexBody`|netcoreapp3.1|4.04μs|3.58ns|13.9ns|0.0499| 0|0|3.69 KB| |#5933|`ObjectExtractorMoreComplexBody`|net472|3.81μs|3.24ns|12.5ns|0.602| 0.0057|0|3.8 KB|

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`EncodeArgs`|net6.0|37.1μs|24.3ns|90.9ns|0.447| 0|0|32.4 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`EncodeArgs`|netcoreapp3.1|54.1μs|24.3ns|94.3ns|0.433| 0|0|32.4 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`EncodeArgs`|net472|65.5μs|35ns|135ns|5.16| 0.0653|0|32.5 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`EncodeLegacyArgs`|net6.0|79.2μs|20.9ns|81ns|0| 0|0|2.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`EncodeLegacyArgs`|netcoreapp3.1|106μs|257ns|994ns|0| 0|0|2.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`EncodeLegacyArgs`|net472|152μs|86.2ns|334ns|0.302| 0|0|2.15 KB| |#5933|`EncodeArgs`|net6.0|38.5μs|20.1ns|77.9ns|0.443| 0|0|32.4 KB| |#5933|`EncodeArgs`|netcoreapp3.1|54.9μs|25ns|96.8ns|0.44| 0|0|32.4 KB| |#5933|`EncodeArgs`|net472|66.4μs|50.2ns|188ns|5.16| 0.0662|0|32.5 KB| |#5933|`EncodeLegacyArgs`|net6.0|76.8μs|441ns|3.27μs|0| 0|0|2.14 KB| |#5933|`EncodeLegacyArgs`|netcoreapp3.1|107μs|343ns|1.33μs|0| 0|0|2.14 KB| |#5933|`EncodeLegacyArgs`|net472|152μs|58.8ns|228ns|0.303| 0|0|2.15 KB|

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`RunWafRealisticBenchmark`|net6.0|185μs|167ns|645ns|0| 0|0|2.42 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`RunWafRealisticBenchmark`|netcoreapp3.1|197μs|344ns|1.33μs|0| 0|0|2.37 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`RunWafRealisticBenchmark`|net472|208μs|82.1ns|318ns|0.311| 0|0|2.43 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`RunWafRealisticBenchmarkWithAttack`|net6.0|122μs|40.6ns|157ns|0| 0|0|1.46 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`RunWafRealisticBenchmarkWithAttack`|netcoreapp3.1|129μs|91.5ns|342ns|0| 0|0|1.45 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`RunWafRealisticBenchmarkWithAttack`|net472|140μs|87.9ns|340ns|0.209| 0|0|1.48 KB| |#5933|`RunWafRealisticBenchmark`|net6.0|185μs|252ns|976ns|0| 0|0|2.42 KB| |#5933|`RunWafRealisticBenchmark`|netcoreapp3.1|198μs|126ns|488ns|0| 0|0|2.37 KB| |#5933|`RunWafRealisticBenchmark`|net472|212μs|73ns|283ns|0.315| 0|0|2.43 KB| |#5933|`RunWafRealisticBenchmarkWithAttack`|net6.0|122μs|52.9ns|205ns|0| 0|0|1.46 KB| |#5933|`RunWafRealisticBenchmarkWithAttack`|netcoreapp3.1|130μs|61.2ns|237ns|0| 0|0|1.45 KB| |#5933|`RunWafRealisticBenchmarkWithAttack`|net472|140μs|101ns|379ns|0.209| 0|0|1.48 KB|

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed :heavy_check_mark: Fewer allocations :tada: #### Fewer allocations :tada: in #5933 | Benchmark | Base Allocated | Diff Allocated | Change | Change % | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 | 265 KB | 262.08 KB | -2.92 KB | -1.10% | Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 | 59.21 KB | 57.3 KB | -1.9 KB | -3.22%

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`StringConcatBenchmark`|net6.0|56.7μs|618ns|6.12μs|0| 0|0|43.44 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`StringConcatBenchmark`|netcoreapp3.1|54.1μs|284ns|1.51μs|0| 0|0|42.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`StringConcatBenchmark`|net472|37.1μs|179ns|714ns|0| 0|0|59.21 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`StringConcatAspectBenchmark`|net6.0|306μs|1.53μs|8.23μs|0| 0|0|254.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`StringConcatAspectBenchmark`|netcoreapp3.1|348μs|1.98μs|13.4μs|0| 0|0|265 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/009c689dff5bce6014c5044ec1cc045abb90435d)|`StringConcatAspectBenchmark`|net472|284μs|5.6μs|54.6μs|0| 0|0|278.53 KB| |#5933|`StringConcatBenchmark`|net6.0|53.1μs|289ns|1.53μs|0| 0|0|43.44 KB| |#5933|`StringConcatBenchmark`|netcoreapp3.1|55.4μs|287ns|1.31μs|0| 0|0|42.64 KB| |#5933|`StringConcatBenchmark`|net472|38.9μs|205ns|1μs|0| 0|0|57.3 KB| |#5933|`StringConcatAspectBenchmark`|net6.0|315μs|1.59μs|9.95μs|0| 0|0|253.73 KB| |#5933|`StringConcatAspectBenchmark`|netcoreapp3.1|335μs|1.57μs|6.86μs|0| 0|0|262.08 KB| |#5933|`StringConcatAspectBenchmark`|net472|280μs|5.72μs|55.4μs|0| 0|0|278.53 KB|

andrewlock commented 2 months ago

Benchmarks Report for tracer :snail:

Benchmarks for #5933 compared to master:

2 benchmarks are faster, with geometric mean 1.136
1 benchmarks are slower, with geometric mean 1.165
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`StartStopWithChild`|net6.0|7.83μs|41.8ns|229ns|0.012| 0.00399|0|5.43 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`StartStopWithChild`|netcoreapp3.1|10.1μs|52.3ns|277ns|0.0247| 0.00987|0|5.62 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`StartStopWithChild`|net472|15.9μs|45.9ns|165ns|1.02| 0.301|0.0949|6.07 KB| |#5933|`StartStopWithChild`|net6.0|7.83μs|44.1ns|286ns|0.0143| 0.00716|0|5.43 KB| |#5933|`StartStopWithChild`|netcoreapp3.1|10μs|55.6ns|334ns|0.0145| 0.00484|0|5.62 KB| |#5933|`StartStopWithChild`|net472|16.5μs|58.4ns|226ns|1.02| 0.295|0.0902|6.07 KB|

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`WriteAndFlushEnrichedTraces`|net6.0|467μs|326ns|1.26μs|0| 0|0|2.7 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|639μs|361ns|1.4μs|0| 0|0|2.7 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`WriteAndFlushEnrichedTraces`|net472|837μs|469ns|1.82μs|0.417| 0|0|3.3 KB| |#5933|`WriteAndFlushEnrichedTraces`|net6.0|505μs|271ns|1.05μs|0| 0|0|2.7 KB| |#5933|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|629μs|299ns|1.16μs|0| 0|0|2.7 KB| |#5933|`WriteAndFlushEnrichedTraces`|net472|837μs|699ns|2.71μs|0.414| 0|0|3.3 KB|

Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`SendRequest`|net6.0|187μs|919ns|4.01μs|0.184| 0|0|18.45 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`SendRequest`|netcoreapp3.1|210μs|1.15μs|8.57μs|0.217| 0|0|20.61 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`SendRequest`|net472|0.000247ns|0.000155ns|0.00056ns|0| 0|0|0 b| |#5933|`SendRequest`|net6.0|195μs|1.11μs|10.7μs|0.188| 0|0|18.45 KB| |#5933|`SendRequest`|netcoreapp3.1|209μs|1.17μs|7.77μs|0.206| 0|0|20.61 KB| |#5933|`SendRequest`|net472|0.000684ns|0.00034ns|0.00132ns|0| 0|0|0 b|

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`WriteAndFlushEnrichedTraces`|net6.0|564μs|2.25μs|8.73μs|0.581| 0|0|41.47 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|670μs|3.11μs|11.6μs|0.342| 0|0|41.61 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`WriteAndFlushEnrichedTraces`|net472|848μs|3.4μs|13.2μs|8.42| 2.66|0.443|53.3 KB| |#5933|`WriteAndFlushEnrichedTraces`|net6.0|564μs|1.5μs|5.39μs|0.573| 0|0|41.6 KB| |#5933|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|653μs|1.53μs|5.71μs|0.322| 0|0|41.77 KB| |#5933|`WriteAndFlushEnrichedTraces`|net472|871μs|3.69μs|13.8μs|8.39| 2.52|0.419|53.29 KB|

Benchmarks.Trace.DbCommandBenchmark - Slower :warning: Same allocations :heavy_check_mark: #### Slower :warning: in #5933 | Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0 | 1.165 | 1,204.01 | 1,402.47 |

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`ExecuteNonQuery`|net6.0|1.2μs|1.04ns|3.89ns|0.0145| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`ExecuteNonQuery`|netcoreapp3.1|1.68μs|1.1ns|4.12ns|0.0133| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`ExecuteNonQuery`|net472|2.02μs|2.08ns|8.06ns|0.156| 0|0|987 B| |#5933|`ExecuteNonQuery`|net6.0|1.4μs|1.1ns|4.12ns|0.0141| 0|0|1.02 KB| |#5933|`ExecuteNonQuery`|netcoreapp3.1|1.65μs|0.815ns|2.94ns|0.0133| 0|0|1.02 KB| |#5933|`ExecuteNonQuery`|net472|2.06μs|2.13ns|7.96ns|0.156| 0|0|987 B|

Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`CallElasticsearch`|net6.0|1.21μs|0.493ns|1.91ns|0.0133| 0|0|976 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`CallElasticsearch`|netcoreapp3.1|1.55μs|0.529ns|1.98ns|0.0132| 0|0|976 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`CallElasticsearch`|net472|2.45μs|1.51ns|5.84ns|0.158| 0|0|995 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`CallElasticsearchAsync`|net6.0|1.27μs|1.24ns|4.62ns|0.0133| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`CallElasticsearchAsync`|netcoreapp3.1|1.6μs|0.662ns|2.48ns|0.0137| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`CallElasticsearchAsync`|net472|2.54μs|2.69ns|10.4ns|0.167| 0|0|1.05 KB| |#5933|`CallElasticsearch`|net6.0|1.17μs|0.756ns|2.93ns|0.0135| 0|0|976 B| |#5933|`CallElasticsearch`|netcoreapp3.1|1.54μs|5.65ns|21.9ns|0.0131| 0|0|976 B| |#5933|`CallElasticsearch`|net472|2.48μs|2.53ns|9.12ns|0.158| 0.00123|0|995 B| |#5933|`CallElasticsearchAsync`|net6.0|1.21μs|0.846ns|3.28ns|0.0134| 0|0|952 B| |#5933|`CallElasticsearchAsync`|netcoreapp3.1|1.67μs|2.47ns|9.25ns|0.0141| 0|0|1.02 KB| |#5933|`CallElasticsearchAsync`|net472|2.51μs|3ns|11.6ns|0.166| 0|0|1.05 KB|

Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`ExecuteAsync`|net6.0|1.22μs|0.768ns|2.97ns|0.0134| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`ExecuteAsync`|netcoreapp3.1|1.66μs|0.944ns|3.65ns|0.0124| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`ExecuteAsync`|net472|1.77μs|4.48ns|17.3ns|0.145| 0|0|915 B| |#5933|`ExecuteAsync`|net6.0|1.25μs|1.14ns|4.41ns|0.0132| 0|0|952 B| |#5933|`ExecuteAsync`|netcoreapp3.1|1.66μs|0.664ns|2.39ns|0.0126| 0|0|952 B| |#5933|`ExecuteAsync`|net472|1.8μs|0.774ns|2.9ns|0.145| 0|0|915 B|

Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`SendAsync`|net6.0|4.33μs|1.55ns|5.57ns|0.0304| 0|0|2.22 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`SendAsync`|netcoreapp3.1|5.13μs|1.3ns|5.03ns|0.0358| 0|0|2.76 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`SendAsync`|net472|7.75μs|1.47ns|5.69ns|0.5| 0|0|3.15 KB| |#5933|`SendAsync`|net6.0|4.15μs|1.67ns|6.27ns|0.0311| 0|0|2.22 KB| |#5933|`SendAsync`|netcoreapp3.1|4.95μs|1.98ns|7.67ns|0.0373| 0|0|2.76 KB| |#5933|`SendAsync`|net472|7.7μs|2.05ns|7.94ns|0.499| 0|0|3.15 KB|

Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|net6.0|1.57μs|0.627ns|2.34ns|0.0229| 0|0|1.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|netcoreapp3.1|2.28μs|0.822ns|2.96ns|0.0219| 0|0|1.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|net472|2.67μs|1.98ns|7.68ns|0.249| 0|0|1.57 KB| |#5933|`EnrichedLog`|net6.0|1.52μs|0.727ns|2.72ns|0.0233| 0|0|1.64 KB| |#5933|`EnrichedLog`|netcoreapp3.1|2.37μs|1.17ns|4.39ns|0.0225| 0|0|1.64 KB| |#5933|`EnrichedLog`|net472|2.71μs|2.99ns|11.6ns|0.249| 0|0|1.57 KB|

Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|net6.0|116μs|218ns|845ns|0| 0|0|4.28 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|netcoreapp3.1|118μs|199ns|717ns|0| 0|0|4.28 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|net472|147μs|254ns|982ns|0.654| 0.218|0|4.46 KB| |#5933|`EnrichedLog`|net6.0|115μs|171ns|662ns|0.0575| 0|0|4.28 KB| |#5933|`EnrichedLog`|netcoreapp3.1|120μs|156ns|541ns|0.0596| 0|0|4.28 KB| |#5933|`EnrichedLog`|net472|147μs|256ns|992ns|0.669| 0.223|0|4.46 KB|

Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|net6.0|3.04μs|0.63ns|2.44ns|0.0305| 0|0|2.2 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|netcoreapp3.1|4.21μs|10.7ns|41.4ns|0.0296| 0|0|2.2 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|net472|5.03μs|2.59ns|10ns|0.32| 0|0|2.02 KB| |#5933|`EnrichedLog`|net6.0|3μs|0.934ns|3.62ns|0.03| 0|0|2.2 KB| |#5933|`EnrichedLog`|netcoreapp3.1|4.08μs|1.34ns|5.2ns|0.0286| 0|0|2.2 KB| |#5933|`EnrichedLog`|net472|4.86μs|1.41ns|5.44ns|0.319| 0|0|2.02 KB|

Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`SendReceive`|net6.0|1.38μs|1.18ns|4.59ns|0.0158| 0|0|1.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`SendReceive`|netcoreapp3.1|1.76μs|0.717ns|2.78ns|0.0149| 0|0|1.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`SendReceive`|net472|2.16μs|2.48ns|9.61ns|0.183| 0.00109|0|1.16 KB| |#5933|`SendReceive`|net6.0|1.36μs|1.16ns|4.47ns|0.0158| 0|0|1.14 KB| |#5933|`SendReceive`|netcoreapp3.1|1.83μs|0.628ns|2.27ns|0.0156| 0|0|1.14 KB| |#5933|`SendReceive`|net472|2.16μs|1.9ns|7.35ns|0.183| 0.00107|0|1.16 KB|

Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|net6.0|2.76μs|1.17ns|4.36ns|0.0221| 0|0|1.6 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|netcoreapp3.1|3.89μs|1.53ns|5.94ns|0.0213| 0|0|1.65 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`EnrichedLog`|net472|4.43μs|2.83ns|11ns|0.323| 0|0|2.04 KB| |#5933|`EnrichedLog`|net6.0|2.79μs|0.709ns|2.74ns|0.0224| 0|0|1.6 KB| |#5933|`EnrichedLog`|netcoreapp3.1|3.95μs|1.41ns|5.45ns|0.0217| 0|0|1.65 KB| |#5933|`EnrichedLog`|net472|4.33μs|2.95ns|11.4ns|0.324| 0|0|2.04 KB|

Benchmarks.Trace.SpanBenchmark - Faster :tada: Same allocations :heavy_check_mark: #### Faster :tada: in #5933 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 | 1.136 | 547.29 | 481.62 |

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`StartFinishSpan`|net6.0|398ns|0.184ns|0.711ns|0.008| 0|0|576 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`StartFinishSpan`|netcoreapp3.1|592ns|0.386ns|1.5ns|0.00778| 0|0|576 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`StartFinishSpan`|net472|653ns|0.175ns|0.676ns|0.0917| 0|0|578 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`StartFinishScope`|net6.0|547ns|0.231ns|0.832ns|0.00956| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`StartFinishScope`|netcoreapp3.1|737ns|0.518ns|2.01ns|0.00918| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`StartFinishScope`|net472|879ns|0.903ns|3.5ns|0.105| 0|0|658 B| |#5933|`StartFinishSpan`|net6.0|385ns|0.163ns|0.608ns|0.00814| 0|0|576 B| |#5933|`StartFinishSpan`|netcoreapp3.1|602ns|1.04ns|4.05ns|0.00769| 0|0|576 B| |#5933|`StartFinishSpan`|net472|600ns|0.858ns|3.32ns|0.0917| 0|0|578 B| |#5933|`StartFinishScope`|net6.0|482ns|0.249ns|0.933ns|0.00968| 0|0|696 B| |#5933|`StartFinishScope`|netcoreapp3.1|764ns|1.01ns|3.93ns|0.00937| 0|0|696 B| |#5933|`StartFinishScope`|net472|856ns|0.547ns|2.12ns|0.104| 0|0|658 B|

Benchmarks.Trace.TraceAnnotationsBenchmark - Faster :tada: Same allocations :heavy_check_mark: #### Faster :tada: in #5933 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 | 1.135 | 1,011.99 | 891.79 |

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`RunOnMethodBegin`|net6.0|647ns|0.413ns|1.6ns|0.0098| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`RunOnMethodBegin`|netcoreapp3.1|1.01μs|0.299ns|1.12ns|0.00962| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/f652e5b92eaefe5495eb43b3e35656f4e2ecde12)|`RunOnMethodBegin`|net472|1.07μs|0.593ns|2.3ns|0.104| 0|0|658 B| |#5933|`RunOnMethodBegin`|net6.0|685ns|0.389ns|1.51ns|0.00968| 0|0|696 B| |#5933|`RunOnMethodBegin`|netcoreapp3.1|892ns|0.661ns|2.38ns|0.00947| 0|0|696 B| |#5933|`RunOnMethodBegin`|net472|1.1μs|0.88ns|3.41ns|0.104| 0|0|658 B|

andrewlock commented 2 months ago

Throughput/Crank Report :zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5933) (11.521M)   : 0, 11521461
    master (11.567M)   : 0, 11566509
    benchmarks/2.9.0 (11.883M)   : 0, 11882814

    section Automatic
    This PR (5933) (7.699M)   : 0, 7698908
    master (7.841M)   : 0, 7841471
    benchmarks/2.9.0 (8.446M)   : 0, 8445613

    section Trace stats
    master (8.125M)   : 0, 8124886

    section Manual
    master (11.708M)   : 0, 11708359

    section Manual + Automatic
    This PR (5933) (7.128M)   : 0, 7128352
    master (7.216M)   : 0, 7216432

    section DD_TRACE_ENABLED=0
    master (10.773M)   : 0, 10773417

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5933) (9.496M)   : 0, 9495914
    master (9.828M)   : 0, 9828340
    benchmarks/2.9.0 (9.760M)   : 0, 9759697

    section Automatic
    This PR (5933) (6.662M)   : 0, 6662455
    master (6.642M)   : 0, 6641935

    section Trace stats
    master (6.912M)   : 0, 6911773

    section Manual
    master (9.501M)   : 0, 9500687

    section Manual + Automatic
    This PR (5933) (6.043M)   : 0, 6042584
    master (6.162M)   : 0, 6161652

    section DD_TRACE_ENABLED=0
    master (8.890M)   : 0, 8889645

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5933) (10.334M)   : 0, 10334401

    section Automatic
    This PR (5933) (6.816M)   : 0, 6816060

    section Manual + Automatic
    This PR (5933) (6.443M)   : 0, 6442724

DataDog / dd-trace-dotnet