Improve logging output of automated package bumps

bouwkast commented 1 month ago

Summary of changes

This is a conglomeration of changes/improvements surrounding auto-package bumps with the main focus of just improving the logging output.

Reason for change

Trying to solve instrumentations potentially out of date and want to make sure that the visibility into potential issues is clear. Coupled with https://github.com/DataDog/dd-trace-dotnet/pull/5908 to fix Dependabot (runs daily) we should be able to keep on top of updates fairly well barring the instrumentations that aren't tested in CI at the moment and require running locally.

Implementation details

Warnings have been added for instrumentations that are not run in CI to call them out as potentially being unsupported.
Added <NoWarn>NU1701</NoWarn> to Honeypot - it isn't a real project so hopefully should be fine
- This shouldn't mute errors such as error NU1202: Package Aerospike.Client 7.2.0 is not compatible with netcoreapp3.1 that were encountered in https://github.com/DataDog/dd-trace-dotnet/pull/5579
Added apm-idm-dotnet as reviewer to the Dependabot Honeypot PRs and to the Automated Test Project PR that happens every Sunday.
Downgraded the max version of CosmosDb in PackageVersionsGeneratorDefinitions.json so that a warning will be added as its tests aren't run in CI. Note: this doesn't change the upper limit of the supported instrumentation.

New warnings from GeneratePackageVersions for potentially out of date instrumentations

[WRN] GeneratePackageVersi: Azure.Messaging.ServiceBus integration may be out of support. The latest version (7.18.1) is greater than last version tested (7.17.5)
[WRN] GeneratePackageVersi: Microsoft.Azure.Cosmos integration may be out of support. The latest version (3.42.0) is greater than last version tested (3.41.0)

New⚠POTENTIALLY UNSUPPORTED⚠ in Datadog.Dependabot.Integrations.csproj for potentially out of date instrumentations

<!-- Integration: Microsoft.Azure.Cosmos.Client -->
<!--    Assembly: Microsoft.Azure.Cosmos.Client -->
<!-- ⚠POTENTIALLY UNSUPPORTED⚠ Latest package https://www.nuget.org/packages/Microsoft.Azure.Cosmos/3.42.0 -->
<PackageReference Include="Microsoft.Azure.Cosmos" Version="3.41.0" />

Test coverage

Ran it locally a lot.
Didn't know really how to test out the GitHub actions and the Dependabot changes though without them on master 🫤

Other details

See https://github.com/DataDog/dd-trace-dotnet/pull/5908 for what should fix Dependabot. Will make tasks to look into CosmosDb and Azure.Messaging.ServiceBus to get clear instructions for IDM on how to test them locally and look into what it'll take to get them into CI.

datadog-ddstaging[bot] commented 1 month ago

Datadog Report

Branch report: steven/auto-package-updating Commit report: b40508f Test service: dd-trace-dotnet

:white_check_mark: 0 Failed, 300946 Passed, 1597 Skipped, 12h 16m 18.37s Total Time

andrewlock commented 1 month ago

Execution-Time Benchmarks Report :stopwatch:

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

andrewlock commented 1 month ago

Benchmarks Report for tracer :snail:

Benchmarks for #5909 compared to master:

4 benchmarks are faster, with geometric mean 1.127
1 benchmarks are slower, with geometric mean 1.207
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`StartStopWithChild`|net6.0|8.01μs|45.4ns|301ns|0.0157| 0.00785|0|5.43 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`StartStopWithChild`|netcoreapp3.1|9.9μs|49.7ns|217ns|0.0146| 0.00485|0|5.62 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`StartStopWithChild`|net472|16.1μs|50.3ns|195ns|1.04| 0.326|0.103|6.06 KB| |#5909|`StartStopWithChild`|net6.0|7.87μs|45ns|351ns|0.011| 0.00365|0|5.43 KB| |#5909|`StartStopWithChild`|netcoreapp3.1|10.3μs|55.1ns|307ns|0.0208| 0.0104|0|5.62 KB| |#5909|`StartStopWithChild`|net472|16.2μs|67.3ns|261ns|1.03| 0.313|0.0964|6.06 KB|

Benchmarks.Trace.AgentWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`WriteAndFlushEnrichedTraces`|net6.0|509μs|344ns|1.24μs|0| 0|0|2.7 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|642μs|237ns|885ns|0| 0|0|2.7 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`WriteAndFlushEnrichedTraces`|net472|861μs|558ns|2.09μs|0.428| 0|0|3.3 KB| |#5909|`WriteAndFlushEnrichedTraces`|net6.0|459μs|106ns|383ns|0| 0|0|2.7 KB| |#5909|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|643μs|476ns|1.65μs|0| 0|0|2.7 KB| |#5909|`WriteAndFlushEnrichedTraces`|net472|836μs|601ns|2.33μs|0.417| 0|0|3.3 KB|

Benchmarks.Trace.AspNetCoreBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`SendRequest`|net6.0|185μs|1.06μs|7.39μs|0.183| 0|0|18.45 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`SendRequest`|netcoreapp3.1|214μs|1.1μs|6.7μs|0.204| 0|0|20.61 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`SendRequest`|net472|0.000732ns|0.000431ns|0.00161ns|0| 0|0|0 b| |#5909|`SendRequest`|net6.0|201μs|1.45μs|14.5μs|0.184| 0|0|18.45 KB| |#5909|`SendRequest`|netcoreapp3.1|216μs|1.11μs|6.77μs|0.204| 0|0|20.61 KB| |#5909|`SendRequest`|net472|9.82E‑06ns|9.82E‑06ns|3.68E‑05ns|0| 0|0|0 b|

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`WriteAndFlushEnrichedTraces`|net6.0|564μs|3.25μs|25.8μs|0.551| 0|0|41.6 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|694μs|3.58μs|16.4μs|0.355| 0|0|41.83 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`WriteAndFlushEnrichedTraces`|net472|865μs|3.58μs|13.9μs|8.13| 2.57|0.428|53.3 KB| |#5909|`WriteAndFlushEnrichedTraces`|net6.0|574μs|1.99μs|7.71μs|0.566| 0|0|41.76 KB| |#5909|`WriteAndFlushEnrichedTraces`|netcoreapp3.1|663μs|3.24μs|13μs|0.329| 0|0|41.78 KB| |#5909|`WriteAndFlushEnrichedTraces`|net472|864μs|3.98μs|15.4μs|8.5| 2.55|0.425|53.29 KB|

Benchmarks.Trace.DbCommandBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`ExecuteNonQuery`|net6.0|1.32μs|1.49ns|5.75ns|0.0144| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`ExecuteNonQuery`|netcoreapp3.1|1.67μs|1.14ns|4.42ns|0.0135| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`ExecuteNonQuery`|net472|2.03μs|1.88ns|7.05ns|0.157| 0|0|987 B| |#5909|`ExecuteNonQuery`|net6.0|1.18μs|1.06ns|4.11ns|0.0143| 0|0|1.02 KB| |#5909|`ExecuteNonQuery`|netcoreapp3.1|1.67μs|1.56ns|5.85ns|0.0135| 0|0|1.02 KB| |#5909|`ExecuteNonQuery`|net472|2μs|1.55ns|5.59ns|0.156| 0|0|987 B|

Benchmarks.Trace.ElasticsearchBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`CallElasticsearch`|net6.0|1.21μs|1.88ns|7.02ns|0.0135| 0|0|976 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`CallElasticsearch`|netcoreapp3.1|1.49μs|0.988ns|3.83ns|0.0135| 0|0|976 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`CallElasticsearch`|net472|2.49μs|1.67ns|6.45ns|0.158| 0|0|995 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`CallElasticsearchAsync`|net6.0|1.35μs|0.614ns|2.3ns|0.0135| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`CallElasticsearchAsync`|netcoreapp3.1|1.63μs|0.668ns|2.5ns|0.0138| 0|0|1.02 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`CallElasticsearchAsync`|net472|2.58μs|3.29ns|12.7ns|0.166| 0|0|1.05 KB| |#5909|`CallElasticsearch`|net6.0|1.25μs|0.568ns|2.12ns|0.0139| 0|0|976 B| |#5909|`CallElasticsearch`|netcoreapp3.1|1.52μs|0.505ns|1.89ns|0.013| 0|0|976 B| |#5909|`CallElasticsearch`|net472|2.43μs|2.72ns|10.5ns|0.158| 0.00123|0|995 B| |#5909|`CallElasticsearchAsync`|net6.0|1.23μs|0.615ns|2.22ns|0.0136| 0|0|952 B| |#5909|`CallElasticsearchAsync`|netcoreapp3.1|1.59μs|0.574ns|2.15ns|0.0135| 0|0|1.02 KB| |#5909|`CallElasticsearchAsync`|net472|2.69μs|1.62ns|5.84ns|0.166| 0.00135|0|1.05 KB|

Benchmarks.Trace.GraphQLBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`ExecuteAsync`|net6.0|1.33μs|1.13ns|4.06ns|0.0134| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`ExecuteAsync`|netcoreapp3.1|1.51μs|0.745ns|2.89ns|0.0127| 0|0|952 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`ExecuteAsync`|net472|1.81μs|1.35ns|5.24ns|0.145| 0|0|915 B| |#5909|`ExecuteAsync`|net6.0|1.24μs|0.73ns|2.83ns|0.013| 0|0|952 B| |#5909|`ExecuteAsync`|netcoreapp3.1|1.63μs|0.602ns|2.25ns|0.0129| 0|0|952 B| |#5909|`ExecuteAsync`|net472|1.7μs|0.628ns|2.43ns|0.145| 0|0|915 B|

Benchmarks.Trace.HttpClientBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`SendAsync`|net6.0|4.19μs|1.91ns|7.41ns|0.0313| 0|0|2.22 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`SendAsync`|netcoreapp3.1|5.06μs|1.56ns|5.83ns|0.0377| 0|0|2.76 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`SendAsync`|net472|7.79μs|2.13ns|8.25ns|0.498| 0|0|3.15 KB| |#5909|`SendAsync`|net6.0|4.1μs|1.36ns|5.1ns|0.0307| 0|0|2.22 KB| |#5909|`SendAsync`|netcoreapp3.1|5.16μs|7.22ns|26ns|0.0362| 0|0|2.76 KB| |#5909|`SendAsync`|net472|7.79μs|3.03ns|11.7ns|0.498| 0|0|3.15 KB|

Benchmarks.Trace.ILoggerBenchmark - Slower :warning: Same allocations :heavy_check_mark: #### Slower :warning: in #5909 | Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 | 1.207 | 2,012.73 | 2,428.65 | #### Faster :tada: in #5909 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net6.0 | 1.123 | 1,637.97 | 1,458.33 |

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|net6.0|1.64μs|0.963ns|3.6ns|0.023| 0|0|1.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|netcoreapp3.1|2.01μs|0.896ns|3.35ns|0.0222| 0|0|1.64 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|net472|2.72μs|2.69ns|10.4ns|0.249| 0|0|1.57 KB| |#5909|`EnrichedLog`|net6.0|1.46μs|0.823ns|3.19ns|0.0225| 0|0|1.64 KB| |#5909|`EnrichedLog`|netcoreapp3.1|2.43μs|1.32ns|5.11ns|0.0218| 0|0|1.64 KB| |#5909|`EnrichedLog`|net472|2.68μs|1.48ns|5.35ns|0.249| 0|0|1.57 KB|

Benchmarks.Trace.Log4netBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|net6.0|115μs|285ns|1.11μs|0.0567| 0|0|4.28 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|netcoreapp3.1|119μs|223ns|833ns|0.0597| 0|0|4.28 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|net472|149μs|155ns|601ns|0.667| 0.222|0|4.46 KB| |#5909|`EnrichedLog`|net6.0|117μs|259ns|1μs|0| 0|0|4.28 KB| |#5909|`EnrichedLog`|netcoreapp3.1|121μs|188ns|677ns|0.0606| 0|0|4.28 KB| |#5909|`EnrichedLog`|net472|150μs|181ns|677ns|0.671| 0.224|0|4.46 KB|

Benchmarks.Trace.NLogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|net6.0|3.08μs|0.724ns|2.71ns|0.0309| 0|0|2.2 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|netcoreapp3.1|4.25μs|1.81ns|7.03ns|0.0297| 0|0|2.2 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|net472|4.83μs|0.936ns|3.38ns|0.319| 0|0|2.02 KB| |#5909|`EnrichedLog`|net6.0|3.08μs|0.636ns|2.46ns|0.031| 0|0|2.2 KB| |#5909|`EnrichedLog`|netcoreapp3.1|4.28μs|1.87ns|7.24ns|0.03| 0|0|2.2 KB| |#5909|`EnrichedLog`|net472|4.76μs|0.671ns|2.51ns|0.32| 0|0|2.02 KB|

Benchmarks.Trace.RedisBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`SendReceive`|net6.0|1.42μs|0.767ns|2.97ns|0.0162| 0|0|1.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`SendReceive`|netcoreapp3.1|1.77μs|1.32ns|4.95ns|0.015| 0|0|1.14 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`SendReceive`|net472|2.2μs|1.69ns|6.53ns|0.184| 0|0|1.16 KB| |#5909|`SendReceive`|net6.0|1.3μs|0.696ns|2.51ns|0.0158| 0|0|1.14 KB| |#5909|`SendReceive`|netcoreapp3.1|1.78μs|0.668ns|2.59ns|0.0151| 0|0|1.14 KB| |#5909|`SendReceive`|net472|2.15μs|1.54ns|5.95ns|0.183| 0.00107|0|1.16 KB|

Benchmarks.Trace.SerilogBenchmark - Same speed :heavy_check_mark: Same allocations :heavy_check_mark:

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|net6.0|2.76μs|1.38ns|4.78ns|0.0221| 0|0|1.6 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|netcoreapp3.1|3.95μs|1.17ns|4.53ns|0.0219| 0|0|1.65 KB| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`EnrichedLog`|net472|4.6μs|1.63ns|6.33ns|0.322| 0|0|2.04 KB| |#5909|`EnrichedLog`|net6.0|2.96μs|1.15ns|4.3ns|0.0222| 0|0|1.6 KB| |#5909|`EnrichedLog`|netcoreapp3.1|3.99μs|1.67ns|6.46ns|0.0221| 0|0|1.65 KB| |#5909|`EnrichedLog`|net472|4.4μs|2.53ns|9.79ns|0.322| 0|0|2.04 KB|

Benchmarks.Trace.SpanBenchmark - Faster :tada: Same allocations :heavy_check_mark: #### Faster :tada: in #5909 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 | 1.140 | 660.13 | 578.83 | | Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 | 1.120 | 859.16 | 766.77 |

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`StartFinishSpan`|net6.0|400ns|0.675ns|2.61ns|0.00802| 0|0|576 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`StartFinishSpan`|netcoreapp3.1|547ns|1.85ns|6.69ns|0.00772| 0|0|576 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`StartFinishSpan`|net472|660ns|0.227ns|0.878ns|0.0916| 0|0|578 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`StartFinishScope`|net6.0|513ns|0.596ns|2.31ns|0.00981| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`StartFinishScope`|netcoreapp3.1|741ns|0.291ns|1.13ns|0.00933| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`StartFinishScope`|net472|859ns|0.632ns|2.36ns|0.104| 0|0|658 B| |#5909|`StartFinishSpan`|net6.0|419ns|0.808ns|3.13ns|0.00814| 0|0|576 B| |#5909|`StartFinishSpan`|netcoreapp3.1|560ns|0.308ns|1.19ns|0.00789| 0|0|576 B| |#5909|`StartFinishSpan`|net472|579ns|0.308ns|1.19ns|0.0916| 0|0|578 B| |#5909|`StartFinishScope`|net6.0|491ns|0.287ns|1.11ns|0.00985| 0|0|696 B| |#5909|`StartFinishScope`|netcoreapp3.1|681ns|0.657ns|2.55ns|0.00978| 0|0|696 B| |#5909|`StartFinishScope`|net472|767ns|0.458ns|1.59ns|0.104| 0|0|658 B|

Benchmarks.Trace.TraceAnnotationsBenchmark - Faster :tada: Same allocations :heavy_check_mark: #### Faster :tada: in #5909 | Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality | |:----------|-----------:|-----------:|--------:|--------:| | Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 | 1.122 | 673.62 | 600.17 |

### Raw results | Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------|---------------------------- |-------------- |---------:|---------:|--------:|-------:|------:|------:|----------:| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`RunOnMethodBegin`|net6.0|673ns|0.659ns|2.38ns|0.00958| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`RunOnMethodBegin`|netcoreapp3.1|942ns|0.427ns|1.65ns|0.00901| 0|0|696 B| |[master](https://github.com/DataDog/dd-trace-dotnet/tree/066e2ef6ea2a8b6638bad8aebb28ce10e063f2b7)|`RunOnMethodBegin`|net472|1.12μs|1.54ns|5.55ns|0.104| 0|0|658 B| |#5909|`RunOnMethodBegin`|net6.0|601ns|0.306ns|1.15ns|0.00971| 0|0|696 B| |#5909|`RunOnMethodBegin`|netcoreapp3.1|861ns|0.864ns|3.35ns|0.0095| 0|0|696 B| |#5909|`RunOnMethodBegin`|net472|1.07μs|0.188ns|0.728ns|0.104| 0|0|658 B|

andrewlock commented 1 month ago

Throughput/Crank Report :zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5909) (11.604M)   : 0, 11604433
    master (11.606M)   : 0, 11606357
    benchmarks/2.9.0 (11.883M)   : 0, 11882814

    section Automatic
    This PR (5909) (7.784M)   : 0, 7783649
    master (7.756M)   : 0, 7755721
    benchmarks/2.9.0 (8.446M)   : 0, 8445613

    section Trace stats
    master (8.110M)   : 0, 8109885

    section Manual
    master (11.466M)   : 0, 11465663

    section Manual + Automatic
    This PR (5909) (7.235M)   : 0, 7234692
    master (7.192M)   : 0, 7192253

    section DD_TRACE_ENABLED=0
    master (10.665M)   : 0, 10665360

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5909) (9.660M)   : 0, 9660320
    benchmarks/2.9.0 (9.760M)   : 0, 9759697

    section Automatic
    This PR (5909) (6.592M)   : 0, 6591602

    section Manual + Automatic
    This PR (5909) (6.264M)   : 0, 6264143

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5909) (10.005M)   : 0, 10005268
    master (10.088M)   : 0, 10088205

    section Automatic
    This PR (5909) (6.693M)   : 0, 6692891
    master (6.620M)   : 0, 6619922

    section Trace stats
    master (7.305M)   : 0, 7304619

    section Manual
    master (9.981M)   : 0, 9980905

    section Manual + Automatic
    This PR (5909) (6.276M)   : 0, 6276161
    master (6.066M)   : 0, 6066177

    section DD_TRACE_ENABLED=0
    master (9.345M)   : 0, 9344849

DataDog / dd-trace-dotnet