Open bouwkast opened 2 months ago
Execution-time results for samples comparing the following branches/commits:
Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.
Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).
gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6010) - mean (71ms) : 68, 73
. : milestone, 71,
master - mean (71ms) : 68, 73
. : milestone, 71,
section CallTarget+Inlining+NGEN
This PR (6010) - mean (1,122ms) : 1097, 1146
. : milestone, 1122,
master - mean (1,117ms) : 1096, 1138
. : milestone, 1117,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6010) - mean (110ms) : 107, 113
. : milestone, 110,
master - mean (110ms) : 107, 114
. : milestone, 110,
section CallTarget+Inlining+NGEN
This PR (6010) - mean (780ms) : 761, 799
. : milestone, 780,
master - mean (778ms) : 763, 793
. : milestone, 778,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6010) - mean (94ms) : 90, 97
. : milestone, 94,
master - mean (93ms) : 90, 96
. : milestone, 93,
section CallTarget+Inlining+NGEN
This PR (6010) - mean (730ms) : 714, 746
. : milestone, 730,
master - mean (735ms) : 716, 753
. : milestone, 735,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6010) - mean (191ms) : 187, 195
. : milestone, 191,
master - mean (191ms) : 188, 193
. : milestone, 191,
section CallTarget+Inlining+NGEN
This PR (6010) - mean (1,202ms) : 1181, 1223
. : milestone, 1202,
master - mean (1,201ms) : 1177, 1225
. : milestone, 1201,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6010) - mean (276ms) : 271, 281
. : milestone, 276,
master - mean (275ms) : 271, 280
. : milestone, 275,
section CallTarget+Inlining+NGEN
This PR (6010) - mean (942ms) : 924, 959
. : milestone, 942,
master - mean (943ms) : 923, 964
. : milestone, 943,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6010) - mean (265ms) : 261, 269
. : milestone, 265,
master - mean (264ms) : 260, 269
. : milestone, 264,
section CallTarget+Inlining+NGEN
This PR (6010) - mean (927ms) : 909, 944
. : milestone, 927,
master - mean (927ms) : 909, 946
. : milestone, 927,
Branch report: steven/AIDM-312
Commit report: 4540146
Test service: dd-trace-dotnet
:x: 6 Failed (1 Known Flaky), 468894 Passed, 3265 Skipped, 28h 55m 43.35s Total Time
This report shows up to 5 failed tests.
WriteAndFlushEnrichedTraces
-Benchmarks.Trace.AgentWriterBenchmark
- Details
SendRequest
- Benchmarks.Trace.AspNetCoreBenchmark
- Details
WriteAndFlushEnrichedTraces
- Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark
- Details
ExecuteNonQuery
- Benchmarks.Trace.DbCommandBenchmark
- Details
ManualAndAutomatic
- Datadog.Trace.ClrProfiler.IntegrationTests.ManualInstrumentationTests
- :snowflake: Known flaky - Details
Throughput results for AspNetCoreSimpleController comparing the following branches/commits:
Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.
Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!
gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6010) (11.204M) : 0, 11204202
master (11.115M) : 0, 11115307
benchmarks/2.9.0 (11.081M) : 0, 11080577
section Automatic
This PR (6010) (7.293M) : 0, 7293026
master (7.335M) : 0, 7334625
benchmarks/2.9.0 (7.732M) : 0, 7732233
section Trace stats
master (7.700M) : 0, 7700429
section Manual
master (10.990M) : 0, 10989510
section Manual + Automatic
This PR (6010) (6.869M) : 0, 6869215
master (6.780M) : 0, 6780487
section DD_TRACE_ENABLED=0
master (10.305M) : 0, 10305060
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6010) (9.531M) : 0, 9530910
master (9.774M) : 0, 9773727
benchmarks/2.9.0 (9.798M) : 0, 9798067
section Automatic
This PR (6010) (6.474M) : 0, 6474248
master (6.463M) : 0, 6462618
section Trace stats
master (6.907M) : 0, 6906626
section Manual
master (9.697M) : 0, 9697280
section Manual + Automatic
This PR (6010) (6.172M) : 0, 6172072
master (6.155M) : 0, 6155472
section DD_TRACE_ENABLED=0
master (8.868M) : 0, 8867528
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6010) (9.775M) : 0, 9775351
master (10.077M) : 0, 10076675
benchmarks/2.9.0 (10.067M) : 0, 10067315
section Automatic
This PR (6010) (6.472M) : 0, 6472450
master (6.567M) : 0, 6566975
benchmarks/2.9.0 (7.552M) : 0, 7552193
section Trace stats
master (7.314M) : 0, 7314135
section Manual
master (10.073M) : 0, 10073436
section Manual + Automatic
This PR (6010) (6.103M) : 0, 6103152
master (6.134M) : 0, 6134127
section DD_TRACE_ENABLED=0
master (9.536M) : 0, 9535909
Benchmarks for #6010 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored.
The following differences have been observed in committed snapshots. It is meant to help the reviewer. The diff is simplistic, so please check some files anyway while we improve it.
2141 occurrences of :
+ component: aspnet,
Summary of changes
Sets
component
tag forASP.NET
toaspnet
Reason for change
component
tag was missing.Implementation details
Add missing tag.
Test coverage
Lots of snapshots
Other details