DataDog / dd-trace-dotnet

.NET Client Library for Datadog APM
https://docs.datadoghq.com/tracing/
Apache License 2.0
437 stars 137 forks source link

[CI] Shorten too long snapshot file names #6024

Closed daniel-romano-DD closed 2 weeks ago

daniel-romano-DD commented 2 weeks ago

Summary of changes

Allows for verify test to override the snapshot file name

Reason for change

Some snapshots file names were too long and difficult to understand

Implementation details

Test coverage

Other details

github-actions[bot] commented 2 weeks ago

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer. The diff is simplistic, so please check some files anyway while we improve it.

datadog-ddstaging[bot] commented 2 weeks ago

Datadog Report

Branch report: dani/asm/shorten_snapshots Commit report: fdb3e9c Test service: dd-trace-dotnet

:white_check_mark: 0 Failed, 364642 Passed, 2058 Skipped, 15h 45m 25.7s Total Time

andrewlock commented 2 weeks ago

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6024) - mean (70ms)  : 66, 74
     .   : milestone, 70,
    master - mean (69ms)  : 67, 71
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6024) - mean (1,114ms)  : 1094, 1133
     .   : milestone, 1114,
    master - mean (1,107ms)  : 1093, 1121
     .   : milestone, 1107,
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6024) - mean (108ms)  : 105, 112
     .   : milestone, 108,
    master - mean (108ms)  : 105, 111
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6024) - mean (815ms)  : 796, 834
     .   : milestone, 815,
    master - mean (819ms)  : 799, 839
     .   : milestone, 819,
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6024) - mean (92ms)  : 90, 95
     .   : milestone, 92,
    master - mean (91ms)  : 88, 95
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6024) - mean (768ms)  : 746, 789
     .   : milestone, 768,
    master - mean (772ms)  : 753, 790
     .   : milestone, 772,
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6024) - mean (191ms)  : 187, 194
     .   : milestone, 191,
    master - mean (191ms)  : 187, 194
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6024) - mean (1,191ms)  : 1167, 1214
     .   : milestone, 1191,
    master - mean (1,192ms)  : 1162, 1221
     .   : milestone, 1192,
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6024) - mean (277ms)  : 273, 281
     .   : milestone, 277,
    master - mean (276ms)  : 271, 281
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6024) - mean (977ms)  : 955, 1000
     .   : milestone, 977,
    master - mean (972ms)  : 950, 994
     .   : milestone, 972,
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6024) - mean (264ms)  : 261, 268
     .   : milestone, 264,
    master - mean (264ms)  : 261, 268
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (6024) - mean (951ms)  : 922, 980
     .   : milestone, 951,
    master - mean (951ms)  : 928, 974
     .   : milestone, 951,
andrewlock commented 2 weeks ago

Benchmarks Report for appsec :snail:

Benchmarks for #6024 compared to master:

The following thresholds were used for comparing the benchmark speeds:

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster :tada: Same allocations :heavy_check_mark: #### Faster :tada: in #6024 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net472 | 1.273 | 210.66 | 165.47 | ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`AllCycleSimpleBody`|net6.0|72.5μs|86.8ns|336ns|0.0729| 0|0|6 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`AllCycleSimpleBody`|netcoreapp3.1|63μs|87.9ns|340ns|0.063| 0|0|6.95 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`AllCycleSimpleBody`|net472|48.3μs|104ns|402ns|1.31| 0|0|8.34 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`AllCycleMoreComplexBody`|net6.0|78.5μs|75.9ns|284ns|0.118| 0|0|9.51 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`AllCycleMoreComplexBody`|netcoreapp3.1|68.8μs|84.9ns|318ns|0.137| 0|0|10.37 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`AllCycleMoreComplexBody`|net472|55.9μs|43.6ns|163ns|1.86| 0.0278|0|11.85 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ObjectExtractorSimpleBody`|net6.0|141ns|0.0796ns|0.298ns|0.00392| 0|0|280 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ObjectExtractorSimpleBody`|netcoreapp3.1|215ns|0.119ns|0.462ns|0.00369| 0|0|272 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ObjectExtractorSimpleBody`|net472|211ns|0.262ns|1.02ns|0.0446| 0|0|281 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ObjectExtractorMoreComplexBody`|net6.0|3.06μs|1.71ns|6.4ns|0.0525| 0|0|3.78 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ObjectExtractorMoreComplexBody`|netcoreapp3.1|3.97μs|1.88ns|7.02ns|0.0492| 0|0|3.69 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ObjectExtractorMoreComplexBody`|net472|3.86μs|2.52ns|9.76ns|0.603| 0.0058|0|3.8 KB| |#6024|`AllCycleSimpleBody`|net6.0|73.6μs|96.8ns|375ns|0.0733| 0|0|6 KB| |#6024|`AllCycleSimpleBody`|netcoreapp3.1|62.2μs|86.7ns|336ns|0.0616| 0|0|6.95 KB| |#6024|`AllCycleSimpleBody`|net472|48.2μs|35.1ns|126ns|1.32| 0|0|8.34 KB| |#6024|`AllCycleMoreComplexBody`|net6.0|77.3μs|102ns|395ns|0.118| 0|0|9.51 KB| |#6024|`AllCycleMoreComplexBody`|netcoreapp3.1|69.7μs|115ns|446ns|0.139| 0|0|10.37 KB| |#6024|`AllCycleMoreComplexBody`|net472|55.5μs|42.3ns|158ns|1.87| 0.0278|0|11.85 KB| |#6024|`ObjectExtractorSimpleBody`|net6.0|139ns|0.112ns|0.403ns|0.00396| 0|0|280 B| |#6024|`ObjectExtractorSimpleBody`|netcoreapp3.1|207ns|0.662ns|2.56ns|0.00359| 0|0|272 B| |#6024|`ObjectExtractorSimpleBody`|net472|165ns|0.15ns|0.581ns|0.0446| 0|0|281 B| |#6024|`ObjectExtractorMoreComplexBody`|net6.0|3.01μs|7.08ns|27.4ns|0.0525| 0|0|3.78 KB| |#6024|`ObjectExtractorMoreComplexBody`|netcoreapp3.1|3.95μs|4.57ns|17.7ns|0.0497| 0|0|3.69 KB| |#6024|`ObjectExtractorMoreComplexBody`|net472|3.82μs|2.18ns|8.45ns|0.602| 0.00573|0|3.8 KB|
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EncodeArgs`|net6.0|38.3μs|30.9ns|120ns|0.441| 0|0|32.4 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EncodeArgs`|netcoreapp3.1|54.3μs|21.7ns|84ns|0.432| 0|0|32.4 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EncodeArgs`|net472|66.7μs|23.3ns|90.3ns|5.16| 0.0666|0|32.5 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EncodeLegacyArgs`|net6.0|71.1μs|28.7ns|111ns|0| 0|0|2.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EncodeLegacyArgs`|netcoreapp3.1|105μs|264ns|1.02μs|0| 0|0|2.15 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EncodeLegacyArgs`|net472|152μs|173ns|670ns|0.306| 0|0|2.15 KB| |#6024|`EncodeArgs`|net6.0|37.4μs|17.4ns|67.2ns|0.45| 0|0|32.4 KB| |#6024|`EncodeArgs`|netcoreapp3.1|54.2μs|27.1ns|105ns|0.433| 0|0|32.4 KB| |#6024|`EncodeArgs`|net472|65.5μs|44.4ns|172ns|5.15| 0.0655|0|32.5 KB| |#6024|`EncodeLegacyArgs`|net6.0|70.9μs|59.9ns|232ns|0| 0|0|2.14 KB| |#6024|`EncodeLegacyArgs`|netcoreapp3.1|104μs|147ns|569ns|0| 0|0|2.14 KB| |#6024|`EncodeLegacyArgs`|net472|152μs|112ns|420ns|0.302| 0|0|2.15 KB|
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`RunWafRealisticBenchmark`|net6.0|183μs|59.1ns|205ns|0| 0|0|2.42 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`RunWafRealisticBenchmark`|netcoreapp3.1|197μs|223ns|862ns|0| 0|0|2.37 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`RunWafRealisticBenchmark`|net472|212μs|157ns|608ns|0.315| 0|0|2.43 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`RunWafRealisticBenchmarkWithAttack`|net6.0|122μs|43.1ns|167ns|0| 0|0|1.46 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`RunWafRealisticBenchmarkWithAttack`|netcoreapp3.1|130μs|266ns|1.03μs|0| 0|0|1.45 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`RunWafRealisticBenchmarkWithAttack`|net472|139μs|34.9ns|135ns|0.211| 0|0|1.48 KB| |#6024|`RunWafRealisticBenchmark`|net6.0|184μs|162ns|628ns|0| 0|0|2.42 KB| |#6024|`RunWafRealisticBenchmark`|netcoreapp3.1|196μs|91.2ns|353ns|0| 0|0|2.37 KB| |#6024|`RunWafRealisticBenchmark`|net472|208μs|52.3ns|196ns|0.314| 0|0|2.43 KB| |#6024|`RunWafRealisticBenchmarkWithAttack`|net6.0|122μs|120ns|415ns|0| 0|0|1.46 KB| |#6024|`RunWafRealisticBenchmarkWithAttack`|netcoreapp3.1|131μs|140ns|541ns|0| 0|0|1.45 KB| |#6024|`RunWafRealisticBenchmarkWithAttack`|net472|139μs|21.6ns|83.7ns|0.208| 0|0|1.48 KB|
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed :heavy_check_mark: More allocations :warning: #### More allocations :warning: in #6024 | Benchmark | Base Allocated | Diff Allocated | Change | Change % | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 | 253.07 KB | 254.91 KB | 1.84 KB | 0.73% ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StringConcatBenchmark`|net6.0|50.9μs|250ns|1.06μs|0| 0|0|43.44 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StringConcatBenchmark`|netcoreapp3.1|53.6μs|292ns|1.55μs|0| 0|0|42.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StringConcatBenchmark`|net472|36.9μs|86.1ns|322ns|0| 0|0|59.67 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StringConcatAspectBenchmark`|net6.0|310μs|1.68μs|9.8μs|0| 0|0|254.52 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StringConcatAspectBenchmark`|netcoreapp3.1|323μs|1.39μs|5.02μs|0| 0|0|253.07 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StringConcatAspectBenchmark`|net472|270μs|5.8μs|56.2μs|0| 0|0|278.53 KB| |#6024|`StringConcatBenchmark`|net6.0|58μs|670ns|6.47μs|0| 0|0|43.44 KB| |#6024|`StringConcatBenchmark`|netcoreapp3.1|59.8μs|1.02μs|10.2μs|0| 0|0|42.64 KB| |#6024|`StringConcatBenchmark`|net472|37μs|105ns|377ns|0| 0|0|59.39 KB| |#6024|`StringConcatAspectBenchmark`|net6.0|277μs|5.93μs|58.4μs|0| 0|0|253.91 KB| |#6024|`StringConcatAspectBenchmark`|netcoreapp3.1|344μs|1.97μs|14.2μs|0| 0|0|254.91 KB| |#6024|`StringConcatAspectBenchmark`|net472|267μs|5.57μs|54μs|0| 0|0|278.53 KB|
andrewlock commented 2 weeks ago

Benchmarks Report for tracer :snail:

Benchmarks for #6024 compared to master:

The following thresholds were used for comparing the benchmark speeds:

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StartStopWithChild`|net6.0|7.5μs|42.3ns|280ns|0.0147| 0.00735|0|5.43 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StartStopWithChild`|netcoreapp3.1|9.85μs|55.7ns|386ns|0.0193| 0.00967|0|5.62 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StartStopWithChild`|net472|15.8μs|40.1ns|155ns|1.03| 0.316|0.0949|6.07 KB| |#6024|`StartStopWithChild`|net6.0|7.46μs|40.4ns|221ns|0.0146| 0.00731|0|5.43 KB| |#6024|`StartStopWithChild`|netcoreapp3.1|9.64μs|44.4ns|172ns|0.0186| 0.00465|0|5.62 KB| |#6024|`StartStopWithChild`|net472|15.9μs|40.7ns|158ns|1.04| 0.317|0.0952|6.06 KB|
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`WriteAndFlushEnrichedTraces`|net6.0|469μs|214ns|801ns|0| 0|0|2.7 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|632μs|369ns|1.38μs|0| 0|0|2.7 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`WriteAndFlushEnrichedTraces`|net472|864μs|473ns|1.83μs|0.428| 0|0|3.3 KB| |#6024|`WriteAndFlushEnrichedTraces`|net6.0|469μs|381ns|1.47μs|0| 0|0|2.7 KB| |#6024|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|625μs|421ns|1.58μs|0| 0|0|2.7 KB| |#6024|`WriteAndFlushEnrichedTraces`|net472|827μs|793ns|2.97μs|0.414| 0|0|3.3 KB|
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`SendRequest`|net6.0|195μs|1.05μs|5.63μs|0.187| 0|0|18.45 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`SendRequest`|netcoreapp3.1|220μs|1.27μs|9.47μs|0.206| 0|0|20.61 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`SendRequest`|net472|0.00797ns|0.000815ns|0.00315ns|0| 0|0|0 b| |#6024|`SendRequest`|net6.0|196μs|1.12μs|7.91μs|0.189| 0|0|18.45 KB| |#6024|`SendRequest`|netcoreapp3.1|222μs|1.27μs|9.94μs|0.203| 0|0|20.61 KB| |#6024|`SendRequest`|net472|0.00399ns|0.00133ns|0.00517ns|0| 0|0|0 b|
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: More allocations :warning: #### More allocations :warning: in #6024 | Benchmark | Base Allocated | Diff Allocated | Change | Change % | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 | 41.62 KB | 41.84 KB | 218 B | 0.52% ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`WriteAndFlushEnrichedTraces`|net6.0|551μs|2.38μs|8.92μs|0.579| 0|0|41.62 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|666μs|2.24μs|8.07μs|0.347| 0|0|41.65 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`WriteAndFlushEnrichedTraces`|net472|865μs|4.12μs|17μs|8.42| 2.66|0.443|53.29 KB| |#6024|`WriteAndFlushEnrichedTraces`|net6.0|579μs|2.78μs|11.8μs|0.566| 0|0|41.84 KB| |#6024|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|680μs|3.19μs|13.1μs|0.342| 0|0|41.75 KB| |#6024|`WriteAndFlushEnrichedTraces`|net472|860μs|4.09μs|16.4μs|8.33| 2.5|0.417|53.28 KB|
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ExecuteNonQuery`|net6.0|1.34μs|1.22ns|4.74ns|0.0144| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ExecuteNonQuery`|netcoreapp3.1|1.83μs|2.61ns|9.77ns|0.0137| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ExecuteNonQuery`|net472|2.09μs|1.33ns|5.13ns|0.157| 0|0|987 B| |#6024|`ExecuteNonQuery`|net6.0|1.26μs|1.28ns|4.97ns|0.0144| 0|0|1.02 KB| |#6024|`ExecuteNonQuery`|netcoreapp3.1|1.71μs|1.62ns|6.26ns|0.0137| 0|0|1.02 KB| |#6024|`ExecuteNonQuery`|net472|1.99μs|1.34ns|5.18ns|0.157| 0|0|987 B|
Benchmarks.Trace.ElasticsearchBenchmark - Faster :tada: Same allocations :heavy_check_mark: #### Faster :tada: in #6024 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 | 1.121 | 1,336.03 | 1,191.62 | ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`CallElasticsearch`|net6.0|1.18μs|1.02ns|3.68ns|0.0136| 0|0|976 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`CallElasticsearch`|netcoreapp3.1|1.58μs|0.986ns|3.69ns|0.0134| 0|0|976 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`CallElasticsearch`|net472|2.48μs|1.56ns|6.05ns|0.158| 0|0|995 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`CallElasticsearchAsync`|net6.0|1.34μs|0.799ns|3.1ns|0.0134| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`CallElasticsearchAsync`|netcoreapp3.1|1.7μs|1.12ns|4.19ns|0.0135| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`CallElasticsearchAsync`|net472|2.66μs|1.3ns|5.02ns|0.166| 0|0|1.05 KB| |#6024|`CallElasticsearch`|net6.0|1.18μs|0.539ns|2.09ns|0.0137| 0|0|976 B| |#6024|`CallElasticsearch`|netcoreapp3.1|1.52μs|2.82ns|10.6ns|0.0129| 0|0|976 B| |#6024|`CallElasticsearch`|net472|2.51μs|1.01ns|3.9ns|0.158| 0|0|995 B| |#6024|`CallElasticsearchAsync`|net6.0|1.19μs|0.516ns|1.93ns|0.0132| 0|0|952 B| |#6024|`CallElasticsearchAsync`|netcoreapp3.1|1.67μs|0.729ns|2.82ns|0.0141| 0|0|1.02 KB| |#6024|`CallElasticsearchAsync`|net472|2.55μs|2.43ns|9.41ns|0.167| 0|0|1.05 KB|
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ExecuteAsync`|net6.0|1.32μs|4.46ns|17.3ns|0.0131| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ExecuteAsync`|netcoreapp3.1|1.63μs|0.623ns|2.33ns|0.0131| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`ExecuteAsync`|net472|1.81μs|0.55ns|2.13ns|0.145| 0|0|915 B| |#6024|`ExecuteAsync`|net6.0|1.31μs|1.32ns|4.92ns|0.0131| 0|0|952 B| |#6024|`ExecuteAsync`|netcoreapp3.1|1.55μs|0.835ns|3.24ns|0.0125| 0|0|952 B| |#6024|`ExecuteAsync`|net472|1.82μs|0.796ns|2.87ns|0.145| 0|0|915 B|
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`SendAsync`|net6.0|4.19μs|0.892ns|3.34ns|0.0315| 0|0|2.22 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`SendAsync`|netcoreapp3.1|5.19μs|2.82ns|10.2ns|0.0362| 0|0|2.76 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`SendAsync`|net472|7.79μs|3.32ns|12.9ns|0.499| 0|0|3.15 KB| |#6024|`SendAsync`|net6.0|4.22μs|1.65ns|5.94ns|0.0315| 0|0|2.22 KB| |#6024|`SendAsync`|netcoreapp3.1|5.09μs|1.86ns|7.18ns|0.0359| 0|0|2.76 KB| |#6024|`SendAsync`|net472|7.86μs|2.21ns|8.57ns|0.498| 0|0|3.15 KB|
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|net6.0|1.57μs|0.545ns|2.04ns|0.0234| 0|0|1.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|netcoreapp3.1|2.29μs|1.07ns|4.14ns|0.0221| 0|0|1.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|net472|2.68μs|2.01ns|7.53ns|0.249| 0|0|1.57 KB| |#6024|`EnrichedLog`|net6.0|1.49μs|0.814ns|3.15ns|0.0233| 0|0|1.64 KB| |#6024|`EnrichedLog`|netcoreapp3.1|2.27μs|0.784ns|2.93ns|0.0227| 0|0|1.64 KB| |#6024|`EnrichedLog`|net472|2.65μs|1.43ns|5.52ns|0.249| 0|0|1.57 KB|
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|net6.0|113μs|191ns|739ns|0.0566| 0|0|4.28 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|netcoreapp3.1|118μs|201ns|777ns|0.0592| 0|0|4.28 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|net472|148μs|166ns|643ns|0.666| 0.222|0|4.46 KB| |#6024|`EnrichedLog`|net6.0|113μs|157ns|606ns|0.0568| 0|0|4.28 KB| |#6024|`EnrichedLog`|netcoreapp3.1|120μs|224ns|866ns|0| 0|0|4.28 KB| |#6024|`EnrichedLog`|net472|147μs|196ns|760ns|0.661| 0.22|0|4.46 KB|
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|net6.0|3.18μs|1ns|3.87ns|0.0297| 0|0|2.2 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|netcoreapp3.1|4.33μs|1.62ns|6.06ns|0.03| 0|0|2.2 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|net472|4.83μs|1.49ns|5.78ns|0.319| 0|0|2.02 KB| |#6024|`EnrichedLog`|net6.0|3.03μs|0.653ns|2.53ns|0.0304| 0|0|2.2 KB| |#6024|`EnrichedLog`|netcoreapp3.1|4.03μs|2.04ns|7.92ns|0.0301| 0|0|2.2 KB| |#6024|`EnrichedLog`|net472|4.8μs|3.69ns|13.8ns|0.319| 0|0|2.02 KB|
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`SendReceive`|net6.0|1.34μs|0.485ns|1.88ns|0.016| 0|0|1.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`SendReceive`|netcoreapp3.1|1.81μs|1.28ns|4.95ns|0.0153| 0|0|1.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`SendReceive`|net472|2.09μs|1.75ns|6.78ns|0.183| 0.00105|0|1.16 KB| |#6024|`SendReceive`|net6.0|1.39μs|1.23ns|4.77ns|0.016| 0|0|1.14 KB| |#6024|`SendReceive`|netcoreapp3.1|1.75μs|0.782ns|2.93ns|0.0149| 0|0|1.14 KB| |#6024|`SendReceive`|net472|2.22μs|1.2ns|4.65ns|0.183| 0.00111|0|1.16 KB|
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|net6.0|2.84μs|1.92ns|7.44ns|0.0226| 0|0|1.6 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|netcoreapp3.1|3.84μs|3.17ns|12.3ns|0.023| 0|0|1.65 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`EnrichedLog`|net472|4.33μs|1.72ns|6.68ns|0.323| 0|0|2.04 KB| |#6024|`EnrichedLog`|net6.0|2.67μs|1.01ns|3.92ns|0.0227| 0|0|1.6 KB| |#6024|`EnrichedLog`|netcoreapp3.1|4.03μs|1.41ns|5.47ns|0.0221| 0|0|1.65 KB| |#6024|`EnrichedLog`|net472|4.35μs|1.71ns|6.62ns|0.322| 0|0|2.04 KB|
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark: #### Slower :warning: in #6024 | Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 | 1.146 | 609.86 | 698.70 | #### Faster :tada: in #6024 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 | 1.153 | 560.42 | 485.88 | ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StartFinishSpan`|net6.0|399ns|0.266ns|1.03ns|0.00816| 0|0|576 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StartFinishSpan`|netcoreapp3.1|609ns|0.818ns|3.17ns|0.00773| 0|0|576 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StartFinishSpan`|net472|610ns|0.648ns|2.51ns|0.0917| 0|0|578 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StartFinishScope`|net6.0|561ns|0.279ns|1.08ns|0.00983| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StartFinishScope`|netcoreapp3.1|677ns|0.951ns|3.68ns|0.00949| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`StartFinishScope`|net472|870ns|1.02ns|3.97ns|0.104| 0|0|658 B| |#6024|`StartFinishSpan`|net6.0|432ns|0.366ns|1.42ns|0.00814| 0|0|576 B| |#6024|`StartFinishSpan`|netcoreapp3.1|593ns|0.297ns|1.15ns|0.00769| 0|0|576 B| |#6024|`StartFinishSpan`|net472|699ns|0.867ns|3.36ns|0.0916| 0|0|578 B| |#6024|`StartFinishScope`|net6.0|485ns|0.384ns|1.49ns|0.00988| 0|0|696 B| |#6024|`StartFinishScope`|netcoreapp3.1|733ns|0.53ns|2.05ns|0.00937| 0|0|696 B| |#6024|`StartFinishScope`|net472|877ns|1.21ns|4.67ns|0.105| 0|0|658 B|
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`RunOnMethodBegin`|net6.0|598ns|0.619ns|2.4ns|0.00983| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`RunOnMethodBegin`|netcoreapp3.1|968ns|0.911ns|3.53ns|0.00926| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/5520a7f5270eda28d33c234c3b91bd5a0a6be186)|`RunOnMethodBegin`|net472|1.1μs|2.03ns|7.86ns|0.104| 0|0|658 B| |#6024|`RunOnMethodBegin`|net6.0|643ns|0.433ns|1.68ns|0.00968| 0|0|696 B| |#6024|`RunOnMethodBegin`|netcoreapp3.1|917ns|1ns|3.88ns|0.00924| 0|0|696 B| |#6024|`RunOnMethodBegin`|net472|1.14μs|1.12ns|4.34ns|0.104| 0|0|658 B|
andrewlock commented 2 weeks ago

Throughput/Crank Report :zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6024) (11.243M)   : 0, 11242699
    master (11.231M)   : 0, 11231191
    benchmarks/2.9.0 (11.382M)   : 0, 11382231

    section Automatic
    This PR (6024) (7.472M)   : 0, 7471660
    master (7.515M)   : 0, 7514608
    benchmarks/2.9.0 (7.834M)   : 0, 7834075

    section Trace stats
    master (7.733M)   : 0, 7732900

    section Manual
    master (11.202M)   : 0, 11202296

    section Manual + Automatic
    This PR (6024) (6.899M)   : 0, 6899112
    master (6.988M)   : 0, 6987518

    section DD_TRACE_ENABLED=0
    master (10.466M)   : 0, 10466013
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6024) (9.591M)   : 0, 9590588
    master (9.544M)   : 0, 9544174
    benchmarks/2.9.0 (9.745M)   : 0, 9744903

    section Automatic
    This PR (6024) (6.632M)   : 0, 6632434
    master (6.633M)   : 0, 6633210

    section Trace stats
    master (6.699M)   : 0, 6699486

    section Manual
    master (9.456M)   : 0, 9455551

    section Manual + Automatic
    This PR (6024) (6.261M)   : 0, 6260766
    master (6.210M)   : 0, 6209519

    section DD_TRACE_ENABLED=0
    master (9.025M)   : 0, 9024862
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6024) (10.080M)   : 0, 10080171
    master (10.276M)   : 0, 10276111
    benchmarks/2.9.0 (10.073M)   : 0, 10073079

    section Automatic
    This PR (6024) (6.732M)   : 0, 6731762
    master (6.636M)   : 0, 6635574
    benchmarks/2.9.0 (7.410M)   : 0, 7409502

    section Trace stats
    master (7.388M)   : 0, 7387724

    section Manual
    master (9.906M)   : 0, 9906408

    section Manual + Automatic
    This PR (6024) (6.078M)   : 0, 6077631
    master (6.175M)   : 0, 6174678

    section DD_TRACE_ENABLED=0
    master (9.259M)   : 0, 9258887