search

DataDog / dd-trace-dotnet

.NET Client Library for Datadog APM
https://docs.datadoghq.com/tracing/
Apache License 2.0
437 stars 137 forks source link

[ASM] Changes to the collection of usr.id for authenticated clients #5738

Closed robertpi closed 1 month ago

robertpi commented 3 months ago

Summary of changes

Change to the automatic instrumentation of common authentication frameworks, to collect details of the authenticated user. This mechanism is only active when ASM is enabled. The full specification is in this internal RFC.

Reason for change

This will improve our detections of Account Take Overs (ATO).

Test coverage

Added two new unit test covering basic functionally such creating a hash to anonymize user ids.

Updated several existing integration tests that covered automatic collection of user ids, and add some integration tests to cover configure user id collection via remote config.

Other details

github-actions[bot] commented 3 months ago

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer. The diff is simplistic, so please check some files anyway while we improve it.

1 occurrences of : 

+      appsec.events.users.login.failure.usr.id: NoSuchUser,

2 occurrences of : 

-      _dd.appsec.events.users.login.failure.auto.mode: safe,
+      _dd.appsec.events.users.login.failure.auto.mode: identification,

1 occurrences of : 

-      _dd.appsec.events.users.login.success.auto.mode: safe,
+      _dd.appsec.events.users.login.success.auto.mode: identification,

1 occurrences of : 

-      appsec.events.users.login.failure.username: NoSuchUser,
[...]
+      appsec.events.users.login.failure.usr.id: NoSuchUser,

2 occurrences of : 

-      _dd.appsec.events.users.login.failure.auto.mode: extended,
+      _dd.appsec.events.users.login.failure.auto.mode: identification,

1 occurrences of : 

-      appsec.events.users.login.failure.email: test@test.com,
[...]
-      appsec.events.users.login.failure.username: TestUser,

1 occurrences of : 

-      usr.email: test@test.com,
[...]
-      usr.name: TestUser,
-      _dd.appsec.events.users.login.success.auto.mode: extended,
+      _dd.appsec.events.users.login.success.auto.mode: identification,
datadog-ddstaging[bot] commented 3 months ago

Datadog Report

Branch report: robert/asm/user-id-collection-modes Commit report: ffadd32 Test service: dd-trace-dotnet

:white_check_mark: 0 Failed, 346392 Passed, 1759 Skipped, 15h 46m 11.62s Total Time

andrewlock commented 3 months ago

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5738) - mean (72ms)  : 64, 80
     .   : milestone, 72,
    master - mean (72ms)  : 64, 79
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (5738) - mean (1,072ms)  : 1043, 1101
     .   : milestone, 1072,
    master - mean (1,072ms)  : 1045, 1098
     .   : milestone, 1072,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5738) - mean (108ms)  : 105, 111
     .   : milestone, 108,
    master - mean (109ms)  : 105, 113
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (5738) - mean (748ms)  : 728, 768
     .   : milestone, 748,
    master - mean (747ms)  : 729, 764
     .   : milestone, 747,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5738) - mean (92ms)  : 89, 96
     .   : milestone, 92,
    master - mean (92ms)  : 89, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (5738) - mean (705ms)  : 684, 725
     .   : milestone, 705,
    master - mean (704ms)  : 677, 730
     .   : milestone, 704,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5738) - mean (192ms)  : 189, 195
     .   : milestone, 192,
    master - mean (192ms)  : 189, 194
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (5738) - mean (1,170ms)  : 1143, 1197
     .   : milestone, 1170,
    master - mean (1,169ms)  : 1144, 1195
     .   : milestone, 1169,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5738) - mean (276ms)  : 272, 280
     .   : milestone, 276,
    master - mean (276ms)  : 272, 281
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (5738) - mean (926ms)  : 902, 950
     .   : milestone, 926,
    master - mean (914ms)  : 896, 933
     .   : milestone, 914,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5738) - mean (267ms)  : 263, 272
     .   : milestone, 267,
    master - mean (265ms)  : 260, 270
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (5738) - mean (908ms)  : 882, 934
     .   : milestone, 908,
    master - mean (901ms)  : 879, 922
     .   : milestone, 901,
andrewlock commented 3 months ago

Benchmarks Report for appsec :snail:

Benchmarks for #5738 compared to master:

The following thresholds were used for comparing the benchmark speeds:

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster :tada: Same allocations :heavy_check_mark: #### Faster :tada: in #5738 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 | 1.144 | 224.29 | 195.98 | ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`AllCycleSimpleBody`|net6.0|72μs|137ns|529ns|0.0717| 0|0|6.01 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`AllCycleSimpleBody`|netcoreapp3.1|63.2μs|89.1ns|345ns|0.0636| 0|0|6.95 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`AllCycleSimpleBody`|net472|49.5μs|87.9ns|340ns|1.31| 0|0|8.34 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`AllCycleMoreComplexBody`|net6.0|78.4μs|68.8ns|258ns|0.119| 0|0|9.51 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`AllCycleMoreComplexBody`|netcoreapp3.1|70.5μs|90.3ns|350ns|0.105| 0|0|10.36 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`AllCycleMoreComplexBody`|net472|56μs|86.9ns|337ns|1.86| 0.0281|0|11.85 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ObjectExtractorSimpleBody`|net6.0|165ns|0.11ns|0.428ns|0.00396| 0|0|280 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ObjectExtractorSimpleBody`|netcoreapp3.1|224ns|0.166ns|0.621ns|0.00382| 0|0|272 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ObjectExtractorSimpleBody`|net472|168ns|0.155ns|0.601ns|0.0446| 0|0|281 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ObjectExtractorMoreComplexBody`|net6.0|3.11μs|1.52ns|5.9ns|0.0528| 0|0|3.78 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ObjectExtractorMoreComplexBody`|netcoreapp3.1|3.93μs|2.83ns|11ns|0.0493| 0|0|3.69 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ObjectExtractorMoreComplexBody`|net472|3.91μs|6.09ns|23.6ns|0.602| 0.00586|0|3.8 KB| |#5738|`AllCycleSimpleBody`|net6.0|73.5μs|75.4ns|292ns|0.0733| 0|0|6 KB| |#5738|`AllCycleSimpleBody`|netcoreapp3.1|62.6μs|114ns|441ns|0.0621| 0|0|6.95 KB| |#5738|`AllCycleSimpleBody`|net472|47.9μs|40.6ns|157ns|1.31| 0|0|8.34 KB| |#5738|`AllCycleMoreComplexBody`|net6.0|79.8μs|48.2ns|180ns|0.119| 0|0|9.51 KB| |#5738|`AllCycleMoreComplexBody`|netcoreapp3.1|70.7μs|150ns|563ns|0.141| 0|0|10.37 KB| |#5738|`AllCycleMoreComplexBody`|net472|56.8μs|59.3ns|230ns|1.87| 0.0284|0|11.85 KB| |#5738|`ObjectExtractorSimpleBody`|net6.0|186ns|0.915ns|3.77ns|0.00392| 0|0|280 B| |#5738|`ObjectExtractorSimpleBody`|netcoreapp3.1|196ns|0.143ns|0.556ns|0.00375| 0|0|272 B| |#5738|`ObjectExtractorSimpleBody`|net472|175ns|0.345ns|1.29ns|0.0446| 0|0|281 B| |#5738|`ObjectExtractorMoreComplexBody`|net6.0|3.05μs|2.72ns|10.2ns|0.0536| 0|0|3.78 KB| |#5738|`ObjectExtractorMoreComplexBody`|netcoreapp3.1|4.07μs|2.61ns|9.4ns|0.0515| 0|0|3.69 KB| |#5738|`ObjectExtractorMoreComplexBody`|net472|3.88μs|4.2ns|16.3ns|0.603| 0.00583|0|3.8 KB|
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EncodeArgs`|net6.0|37.4μs|15.8ns|61.1ns|0.447| 0|0|32.4 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EncodeArgs`|netcoreapp3.1|54.3μs|16.5ns|59.5ns|0.432| 0|0|32.4 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EncodeArgs`|net472|66.9μs|31.5ns|122ns|5.13| 0.0667|0|32.5 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EncodeLegacyArgs`|net6.0|74.8μs|391ns|1.92μs|0.0351| 0|0|2.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EncodeLegacyArgs`|netcoreapp3.1|104μs|61.2ns|212ns|0| 0|0|2.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EncodeLegacyArgs`|net472|155μs|98.8ns|383ns|0.309| 0|0|2.15 KB| |#5738|`EncodeArgs`|net6.0|37.5μs|28ns|105ns|0.451| 0|0|32.4 KB| |#5738|`EncodeArgs`|netcoreapp3.1|54.6μs|19.9ns|74.3ns|0.439| 0|0|32.4 KB| |#5738|`EncodeArgs`|net472|67.4μs|37.9ns|147ns|5.16| 0.067|0|32.5 KB| |#5738|`EncodeLegacyArgs`|net6.0|78.1μs|44.2ns|171ns|0| 0|0|2.14 KB| |#5738|`EncodeLegacyArgs`|netcoreapp3.1|105μs|112ns|433ns|0| 0|0|2.14 KB| |#5738|`EncodeLegacyArgs`|net472|154μs|100ns|388ns|0.309| 0|0|2.15 KB|
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`RunWafRealisticBenchmark`|net6.0|185μs|64.3ns|249ns|0| 0|0|2.42 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`RunWafRealisticBenchmark`|netcoreapp3.1|197μs|92.8ns|347ns|0| 0|0|2.37 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`RunWafRealisticBenchmark`|net472|213μs|192ns|744ns|0.315| 0|0|2.43 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`RunWafRealisticBenchmarkWithAttack`|net6.0|122μs|68.9ns|258ns|0| 0|0|1.46 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`RunWafRealisticBenchmarkWithAttack`|netcoreapp3.1|132μs|134ns|500ns|0| 0|0|1.45 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`RunWafRealisticBenchmarkWithAttack`|net472|139μs|46.2ns|179ns|0.208| 0|0|1.48 KB| |#5738|`RunWafRealisticBenchmark`|net6.0|185μs|358ns|1.39μs|0| 0|0|2.42 KB| |#5738|`RunWafRealisticBenchmark`|netcoreapp3.1|195μs|234ns|908ns|0| 0|0|2.37 KB| |#5738|`RunWafRealisticBenchmark`|net472|210μs|171ns|664ns|0.314| 0|0|2.43 KB| |#5738|`RunWafRealisticBenchmarkWithAttack`|net6.0|122μs|117ns|453ns|0| 0|0|1.46 KB| |#5738|`RunWafRealisticBenchmarkWithAttack`|netcoreapp3.1|130μs|86.6ns|324ns|0| 0|0|1.45 KB| |#5738|`RunWafRealisticBenchmarkWithAttack`|net472|140μs|79.4ns|286ns|0.211| 0|0|1.48 KB|
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed :heavy_check_mark: More allocations :warning: #### More allocations :warning: in #5738 | Benchmark | Base Allocated | Diff Allocated | Change | Change % | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 | 252.67 KB | 266.77 KB | 14.1 KB | 5.58% #### Fewer allocations :tada: in #5738 | Benchmark | Base Allocated | Diff Allocated | Change | Change % | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 | 59.22 KB | 57.16 KB | -2.06 KB | -3.49% | Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 | 264.21 KB | 253.93 KB | -10.28 KB | -3.89% ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StringConcatBenchmark`|net6.0|62.2μs|774ns|7.7μs|0| 0|0|43.44 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StringConcatBenchmark`|netcoreapp3.1|61μs|755ns|7.44μs|0| 0|0|42.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StringConcatBenchmark`|net472|37.8μs|114ns|427ns|0| 0|0|59.22 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StringConcatAspectBenchmark`|net6.0|301μs|1.72μs|13.4μs|0| 0|0|252.67 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StringConcatAspectBenchmark`|netcoreapp3.1|349μs|1.81μs|10.7μs|0| 0|0|264.21 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StringConcatAspectBenchmark`|net472|283μs|5.91μs|57.9μs|0| 0|0|278.53 KB| |#5738|`StringConcatBenchmark`|net6.0|63.1μs|809ns|8.09μs|0| 0|0|43.44 KB| |#5738|`StringConcatBenchmark`|netcoreapp3.1|54.4μs|303ns|1.82μs|0| 0|0|42.64 KB| |#5738|`StringConcatBenchmark`|net472|39.2μs|210ns|1.11μs|0| 0|0|57.16 KB| |#5738|`StringConcatAspectBenchmark`|net6.0|317μs|6.4μs|63.4μs|0| 0|0|266.77 KB| |#5738|`StringConcatAspectBenchmark`|netcoreapp3.1|336μs|1.88μs|12.2μs|0| 0|0|253.93 KB| |#5738|`StringConcatAspectBenchmark`|net472|277μs|5.3μs|51.7μs|0| 0|0|278.53 KB|
andrewlock commented 3 months ago

Benchmarks Report for tracer :snail:

Benchmarks for #5738 compared to master:

The following thresholds were used for comparing the benchmark speeds:

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StartStopWithChild`|net6.0|7.63μs|42.1ns|263ns|0.0153| 0.00765|0|5.43 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StartStopWithChild`|netcoreapp3.1|9.84μs|39.6ns|137ns|0.0145| 0.00484|0|5.62 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StartStopWithChild`|net472|16μs|62.2ns|241ns|1.02| 0.3|0.0948|6.07 KB| |#5738|`StartStopWithChild`|net6.0|7.87μs|43.4ns|345ns|0.0152| 0.00759|0|5.42 KB| |#5738|`StartStopWithChild`|netcoreapp3.1|9.92μs|54.5ns|345ns|0.0149| 0.00496|0|5.61 KB| |#5738|`StartStopWithChild`|net472|16μs|66.9ns|259ns|1.01| 0.304|0.0879|6.08 KB|
Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`WriteAndFlushEnrichedTraces`|net6.0|456μs|448ns|1.74μs|0| 0|0|2.7 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|630μs|435ns|1.69μs|0| 0|0|2.7 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`WriteAndFlushEnrichedTraces`|net472|831μs|740ns|2.77μs|0.411| 0|0|3.3 KB| |#5738|`WriteAndFlushEnrichedTraces`|net6.0|505μs|379ns|1.36μs|0| 0|0|2.7 KB| |#5738|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|637μs|356ns|1.38μs|0| 0|0|2.7 KB| |#5738|`WriteAndFlushEnrichedTraces`|net472|865μs|765ns|2.86μs|0.431| 0|0|3.3 KB|
Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`SendRequest`|net6.0|212μs|1.21μs|8.71μs|0.203| 0|0|18.45 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`SendRequest`|netcoreapp3.1|224μs|1.19μs|6.53μs|0.242| 0|0|20.61 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`SendRequest`|net472|0.00427ns|0.00176ns|0.00682ns|0| 0|0|0 b| |#5738|`SendRequest`|net6.0|199μs|1.18μs|11.8μs|0.185| 0|0|18.45 KB| |#5738|`SendRequest`|netcoreapp3.1|238μs|1.37μs|10.4μs|0.225| 0|0|20.61 KB| |#5738|`SendRequest`|net472|0.00283ns|0.000872ns|0.00338ns|0| 0|0|0 b|
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`WriteAndFlushEnrichedTraces`|net6.0|581μs|3.18μs|19.3μs|0.566| 0|0|41.62 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|703μs|3.96μs|37.6μs|0.34| 0|0|41.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`WriteAndFlushEnrichedTraces`|net472|864μs|4.05μs|15.2μs|8.42| 2.66|0.443|53.31 KB| |#5738|`WriteAndFlushEnrichedTraces`|net6.0|591μs|3.14μs|16μs|0.558| 0|0|41.57 KB| |#5738|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|735μs|4.11μs|25.7μs|0.383| 0|0|41.69 KB| |#5738|`WriteAndFlushEnrichedTraces`|net472|892μs|3.3μs|12.8μs|8.04| 2.23|0.446|53.31 KB|
Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ExecuteNonQuery`|net6.0|1.19μs|1.18ns|4.41ns|0.0143| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ExecuteNonQuery`|netcoreapp3.1|1.76μs|1.78ns|6.66ns|0.0132| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ExecuteNonQuery`|net472|2.01μs|4.05ns|15.7ns|0.156| 0|0|987 B| |#5738|`ExecuteNonQuery`|net6.0|1.29μs|1.14ns|4.26ns|0.014| 0|0|1.02 KB| |#5738|`ExecuteNonQuery`|netcoreapp3.1|1.77μs|1.74ns|6.74ns|0.0133| 0|0|1.02 KB| |#5738|`ExecuteNonQuery`|net472|2μs|1.66ns|6.2ns|0.156| 0|0|987 B|
Benchmarks.Trace.ElasticsearchBenchmark - Slower :warning: Same allocations :heavy_check_mark: #### Slower :warning: in #5738 | Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 | 1.118 | 1,114.68 | 1,246.21 | ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`CallElasticsearch`|net6.0|1.12μs|0.476ns|1.78ns|0.0135| 0|0|976 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`CallElasticsearch`|netcoreapp3.1|1.5μs|0.553ns|2.14ns|0.0128| 0|0|976 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`CallElasticsearch`|net472|2.48μs|1.05ns|4.08ns|0.157| 0|0|995 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`CallElasticsearchAsync`|net6.0|1.32μs|0.756ns|2.83ns|0.0131| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`CallElasticsearchAsync`|netcoreapp3.1|1.65μs|0.94ns|3.52ns|0.014| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`CallElasticsearchAsync`|net472|2.7μs|1.74ns|6.72ns|0.166| 0|0|1.05 KB| |#5738|`CallElasticsearch`|net6.0|1.25μs|0.534ns|2ns|0.0134| 0|0|976 B| |#5738|`CallElasticsearch`|netcoreapp3.1|1.47μs|0.823ns|3.19ns|0.0133| 0|0|976 B| |#5738|`CallElasticsearch`|net472|2.4μs|3.34ns|12.9ns|0.157| 0|0|995 B| |#5738|`CallElasticsearchAsync`|net6.0|1.3μs|1.21ns|4.67ns|0.013| 0|0|952 B| |#5738|`CallElasticsearchAsync`|netcoreapp3.1|1.61μs|1.35ns|5.24ns|0.0137| 0|0|1.02 KB| |#5738|`CallElasticsearchAsync`|net472|2.65μs|2.24ns|8.67ns|0.166| 0|0|1.05 KB|
Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ExecuteAsync`|net6.0|1.15μs|0.745ns|2.79ns|0.0132| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ExecuteAsync`|netcoreapp3.1|1.57μs|0.965ns|3.74ns|0.0126| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`ExecuteAsync`|net472|1.75μs|1.8ns|6.96ns|0.145| 0|0|915 B| |#5738|`ExecuteAsync`|net6.0|1.23μs|0.707ns|2.74ns|0.0136| 0|0|952 B| |#5738|`ExecuteAsync`|netcoreapp3.1|1.67μs|0.802ns|3.11ns|0.0125| 0|0|952 B| |#5738|`ExecuteAsync`|net472|1.77μs|0.463ns|1.79ns|0.145| 0|0|915 B|
Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`SendAsync`|net6.0|4.3μs|1.61ns|6.04ns|0.0299| 0|0|2.22 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`SendAsync`|netcoreapp3.1|5.09μs|2.03ns|7.6ns|0.0355| 0|0|2.76 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`SendAsync`|net472|7.75μs|3.5ns|13.6ns|0.499| 0|0|3.15 KB| |#5738|`SendAsync`|net6.0|4.28μs|1.81ns|6.76ns|0.0321| 0|0|2.22 KB| |#5738|`SendAsync`|netcoreapp3.1|5.13μs|2.18ns|8.42ns|0.0358| 0|0|2.76 KB| |#5738|`SendAsync`|net472|7.69μs|5.64ns|21.1ns|0.499| 0|0|3.15 KB|
Benchmarks.Trace.ILoggerBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|net6.0|1.56μs|1.15ns|4.47ns|0.0234| 0|0|1.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|netcoreapp3.1|2.21μs|3.89ns|15ns|0.023| 0|0|1.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|net472|2.69μs|1.23ns|4.6ns|0.249| 0|0|1.57 KB| |#5738|`EnrichedLog`|net6.0|1.44μs|0.804ns|3.01ns|0.0231| 0|0|1.64 KB| |#5738|`EnrichedLog`|netcoreapp3.1|2.26μs|1.22ns|4.42ns|0.0223| 0|0|1.64 KB| |#5738|`EnrichedLog`|net472|2.52μs|1.26ns|4.87ns|0.249| 0|0|1.57 KB|
Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|net6.0|117μs|167ns|646ns|0.059| 0|0|4.28 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|netcoreapp3.1|120μs|205ns|794ns|0| 0|0|4.28 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|net472|150μs|175ns|678ns|0.672| 0.224|0|4.46 KB| |#5738|`EnrichedLog`|net6.0|115μs|234ns|907ns|0.0571| 0|0|4.28 KB| |#5738|`EnrichedLog`|netcoreapp3.1|120μs|154ns|596ns|0| 0|0|4.28 KB| |#5738|`EnrichedLog`|net472|147μs|178ns|688ns|0.663| 0.221|0|4.46 KB|
Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|net6.0|2.83μs|1.18ns|4.56ns|0.0311| 0|0|2.2 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|netcoreapp3.1|4.12μs|1.78ns|6.41ns|0.0288| 0|0|2.2 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|net472|4.81μs|3.49ns|12.1ns|0.318| 0|0|2.02 KB| |#5738|`EnrichedLog`|net6.0|3.06μs|0.558ns|2.16ns|0.0307| 0|0|2.2 KB| |#5738|`EnrichedLog`|netcoreapp3.1|4.1μs|2.09ns|8.11ns|0.0287| 0|0|2.2 KB| |#5738|`EnrichedLog`|net472|4.87μs|1.18ns|4.55ns|0.32| 0|0|2.02 KB|
Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`SendReceive`|net6.0|1.34μs|0.534ns|2ns|0.0161| 0|0|1.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`SendReceive`|netcoreapp3.1|1.74μs|0.866ns|3.24ns|0.015| 0|0|1.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`SendReceive`|net472|2.13μs|0.731ns|2.73ns|0.183| 0|0|1.16 KB| |#5738|`SendReceive`|net6.0|1.37μs|1.29ns|4.99ns|0.0164| 0|0|1.14 KB| |#5738|`SendReceive`|netcoreapp3.1|1.73μs|0.744ns|2.88ns|0.0155| 0|0|1.14 KB| |#5738|`SendReceive`|net472|2.1μs|0.873ns|3.38ns|0.183| 0.00105|0|1.16 KB|
Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|net6.0|2.61μs|0.818ns|2.95ns|0.0221| 0|0|1.6 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|netcoreapp3.1|3.85μs|2.62ns|9.46ns|0.0213| 0|0|1.65 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`EnrichedLog`|net472|4.41μs|2.77ns|10.4ns|0.323| 0|0|2.04 KB| |#5738|`EnrichedLog`|net6.0|2.7μs|1.16ns|4.5ns|0.0216| 0|0|1.6 KB| |#5738|`EnrichedLog`|netcoreapp3.1|3.95μs|1.14ns|4.11ns|0.0217| 0|0|1.65 KB| |#5738|`EnrichedLog`|net472|4.34μs|3.42ns|13.3ns|0.323| 0|0|2.04 KB|
Benchmarks.Trace.SpanBenchmark - Slower :warning: Same allocations :heavy_check_mark: #### Slower :warning: in #5738 | Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 | 1.139 | 550.70 | 627.17 | ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StartFinishSpan`|net6.0|396ns|0.239ns|0.927ns|0.00818| 0|0|576 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StartFinishSpan`|netcoreapp3.1|551ns|0.495ns|1.79ns|0.0077| 0|0|576 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StartFinishSpan`|net472|636ns|0.44ns|1.7ns|0.0917| 0|0|578 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StartFinishScope`|net6.0|484ns|0.263ns|1.02ns|0.00986| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StartFinishScope`|netcoreapp3.1|715ns|0.365ns|1.41ns|0.00932| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`StartFinishScope`|net472|814ns|0.662ns|2.56ns|0.104| 0|0|658 B| |#5738|`StartFinishSpan`|net6.0|397ns|0.482ns|1.87ns|0.00814| 0|0|576 B| |#5738|`StartFinishSpan`|netcoreapp3.1|629ns|1.98ns|7.66ns|0.00744| 0|0|576 B| |#5738|`StartFinishSpan`|net472|691ns|0.693ns|2.68ns|0.0918| 0|0|578 B| |#5738|`StartFinishScope`|net6.0|475ns|0.186ns|0.671ns|0.00977| 0|0|696 B| |#5738|`StartFinishScope`|netcoreapp3.1|770ns|0.506ns|1.89ns|0.00934| 0|0|696 B| |#5738|`StartFinishScope`|net472|851ns|0.471ns|1.83ns|0.104| 0|0|658 B|
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark: ### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`RunOnMethodBegin`|net6.0|674ns|0.387ns|1.5ns|0.00986| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`RunOnMethodBegin`|netcoreapp3.1|921ns|0.729ns|2.82ns|0.00927| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/dc1a21fb047cc19652432e14422a026edb66fc2f)|`RunOnMethodBegin`|net472|1.08μs|0.482ns|1.87ns|0.104| 0|0|658 B| |#5738|`RunOnMethodBegin`|net6.0|662ns|0.157ns|0.609ns|0.00959| 0|0|696 B| |#5738|`RunOnMethodBegin`|netcoreapp3.1|933ns|0.225ns|0.841ns|0.00938| 0|0|696 B| |#5738|`RunOnMethodBegin`|net472|1.09μs|0.27ns|1.05ns|0.104| 0|0|658 B|
andrewlock commented 3 months ago

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5738) (10.839M)   : 0, 10839499
    master (11.705M)   : 0, 11705096
    benchmarks/2.9.0 (12.028M)   : 0, 12027804

    section Automatic
    This PR (5738) (7.464M)   : 0, 7464442
    master (7.833M)   : 0, 7832844
    benchmarks/2.9.0 (8.471M)   : 0, 8471115

    section Trace stats
    master (8.082M)   : 0, 8081964

    section Manual
    This PR (5738) (9.710M)   : 0, 9710455
    master (9.998M)   : 0, 9998100

    section Manual + Automatic
    This PR (5738) (7.135M)   : 0, 7135008
    master (7.346M)   : 0, 7345601

    section Version Conflict
    master (6.643M)   : 0, 6642934

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5738) (9.689M)   : 0, 9688856
    master (9.535M)   : 0, 9534826
    benchmarks/2.9.0 (9.556M)   : 0, 9555936

    section Automatic
    This PR (5738) (6.701M)   : 0, 6701428
    master (6.547M)   : 0, 6546581

    section Trace stats
    master (6.947M)   : 0, 6947401

    section Manual
    This PR (5738) (8.318M)   : 0, 8317505
    master (8.293M)   : 0, 8292792

    section Manual + Automatic
    This PR (5738) (6.207M)   : 0, 6207003
    master (6.185M)   : 0, 6184603

    section Version Conflict
    master (5.716M)   : 0, 5716316

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5738) (10.101M)   : 0, 10100755
    master (10.415M)   : 0, 10414832
    benchmarks/2.9.0 (9.885M)   : 0, 9885158

    section Automatic
    This PR (5738) (7.178M)   : 0, 7177794
    master (7.218M)   : 0, 7218121
    benchmarks/2.9.0 (6.977M)   : 0, 6977395

    section Trace stats
    master (7.576M)   : 0, 7576347

    section Manual
    This PR (5738) (8.951M)   : 0, 8950838
    master (9.045M)   : 0, 9045472

    section Manual + Automatic
    This PR (5738) (6.952M)   : 0, 6951664
    master (6.817M)   : 0, 6816758

    section Version Conflict
    master (6.066M)   : 0, 6066034
datadog-ddstaging[bot] commented 2 months ago

Datadog Report

Branch report: robert/asm/user-id-collection-modes Commit report: be2cd28 Test service: dd-trace-dotnet

:x: 8 Failed (0 Known Flaky), 346276 Passed, 1696 Skipped, 14h 27m 12.63s Total Time

:x: Failed Tests (8)

This report shows up to 5 failed tests.

  • TestChangeUserIdCollection - Datadog.Trace.Security.IntegrationTests.Rcm.AspNetCore5AsmFeatureUserIdSecurityEnabled - Details
<details>
<summary>Expand for error</summary>

```
Results do not match.
Differences:
Received: Security.AspNetCore5AsmFeatureUserIdSecurityEnabled._.received.txt
Verified: Security.AspNetCore5AsmFeatureUserIdSecurityEnabled._.verified.txt
Received Content:
[
  {
    TraceId: Id_1,
    SpanId: Id_2,
    Name: aspnet_core.request,
...
```
</details>
datadog-ddstaging[bot] commented 2 months ago

Datadog Report

Branch report: robert/asm/user-id-collection-modes Commit report: be2cd28 Test service: dd-trace-dotnet

:x: 8 Failed (0 Known Flaky), 346276 Passed, 1696 Skipped, 14h 27m 12.63s Total Time

:x: Failed Tests (8)

This report shows up to 5 failed tests.

  • TestChangeUserIdCollection - Datadog.Trace.Security.IntegrationTests.Rcm.AspNetCore5AsmFeatureUserIdSecurityEnabled - Details
<details>
<summary>Expand for error</summary>

```
Results do not match.
Differences:
Received: Security.AspNetCore5AsmFeatureUserIdSecurityEnabled._.received.txt
Verified: Security.AspNetCore5AsmFeatureUserIdSecurityEnabled._.verified.txt
Received Content:
[
  {
    TraceId: Id_1,
    SpanId: Id_2,
    Name: aspnet_core.request,
...
```
</details>
datadog-ddstaging[bot] commented 2 months ago

Datadog Report

Branch report: robert/asm/user-id-collection-modes Commit report: be2cd28 Test service: dd-trace-dotnet

:x: 8 Failed (0 Known Flaky), 346276 Passed, 1696 Skipped, 14h 27m 12.63s Total Time

:x: Failed Tests (8)

This report shows up to 5 failed tests.

  • TestChangeUserIdCollection - Datadog.Trace.Security.IntegrationTests.Rcm.AspNetCore5AsmFeatureUserIdSecurityEnabled - Details
<details>
<summary>Expand for error</summary>

```
Results do not match.
Differences:
Received: Security.AspNetCore5AsmFeatureUserIdSecurityEnabled._.received.txt
Verified: Security.AspNetCore5AsmFeatureUserIdSecurityEnabled._.verified.txt
Received Content:
[
  {
    TraceId: Id_1,
    SpanId: Id_2,
    Name: aspnet_core.request,
...
```
</details>
datadog-ddstaging[bot] commented 2 months ago

Datadog Report

Branch report: robert/asm/user-id-collection-modes Commit report: be2cd28 Test service: dd-trace-dotnet

:x: 8 Failed (0 Known Flaky), 346276 Passed, 1696 Skipped, 14h 27m 12.63s Total Time

:x: Failed Tests (8)

This report shows up to 5 failed tests.

  • TestChangeUserIdCollection - Datadog.Trace.Security.IntegrationTests.Rcm.AspNetCore5AsmFeatureUserIdSecurityEnabled - Details
<details>
<summary>Expand for error</summary>

```
Results do not match.
Differences:
Received: Security.AspNetCore5AsmFeatureUserIdSecurityEnabled._.received.txt
Verified: Security.AspNetCore5AsmFeatureUserIdSecurityEnabled._.verified.txt
Received Content:
[
  {
    TraceId: Id_1,
    SpanId: Id_2,
    Name: aspnet_core.request,
...
```
</details>