Closed andrewlock closed 3 weeks ago
datadog-ddstaging[bot]
commented
3 weeks ago Branch report: andrew/ci/fixes
Commit report: d97d464
Test service: dd-trace-dotnet
:white_check_mark: 0 Failed, 364768 Passed, 2049 Skipped, 15h 2m 1.74s Total Time
andrewlock
commented
3 weeks ago Execution-time results for samples comparing the following branches/commits:
Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.
Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).
gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6000) - mean (71ms) : 67, 75
. : milestone, 71,
master - mean (70ms) : 66, 74
. : milestone, 70,
section CallTarget+Inlining+NGEN
This PR (6000) - mean (1,081ms) : 1060, 1102
. : milestone, 1081,
master - mean (1,084ms) : 1061, 1107
. : milestone, 1084,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6000) - mean (109ms) : 106, 112
. : milestone, 109,
master - mean (109ms) : 105, 112
. : milestone, 109,
section CallTarget+Inlining+NGEN
This PR (6000) - mean (765ms) : 744, 785
. : milestone, 765,
master - mean (763ms) : 740, 786
. : milestone, 763,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6000) - mean (93ms) : 89, 96
. : milestone, 93,
master - mean (93ms) : 89, 97
. : milestone, 93,
section CallTarget+Inlining+NGEN
This PR (6000) - mean (714ms) : 694, 734
. : milestone, 714,
master - mean (714ms) : 694, 733
. : milestone, 714,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6000) - mean (190ms) : 187, 194
. : milestone, 190,
master - mean (191ms) : 188, 195
. : milestone, 191,
section CallTarget+Inlining+NGEN
This PR (6000) - mean (1,163ms) : 1132, 1194
. : milestone, 1163,
master - mean (1,165ms) : 1136, 1193
. : milestone, 1165,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6000) - mean (274ms) : 269, 280
. : milestone, 274,
master - mean (275ms) : 271, 279
. : milestone, 275,
section CallTarget+Inlining+NGEN
This PR (6000) - mean (919ms) : 896, 943
. : milestone, 919,
master - mean (923ms) : 898, 948
. : milestone, 923,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6000) - mean (265ms) : 261, 268
. : milestone, 265,
master - mean (265ms) : 260, 270
. : milestone, 265,
section CallTarget+Inlining+NGEN
This PR (6000) - mean (903ms) : 883, 924
. : milestone, 903,
master - mean (906ms) : 884, 929
. : milestone, 906,
Throughput results for AspNetCoreSimpleController comparing the following branches/commits:
Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.
Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!
gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6000) (11.325M) : 0, 11325490
master (11.181M) : 0, 11180988
section Automatic
This PR (6000) (7.472M) : 0, 7472369
master (7.344M) : 0, 7344076
section Trace stats
master (7.631M) : 0, 7630555
section Manual
master (11.084M) : 0, 11084343
section Manual + Automatic
This PR (6000) (6.921M) : 0, 6920765
master (6.749M) : 0, 6748775
section DD_TRACE_ENABLED=0
master (10.330M) : 0, 10330053
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6000) (7.383M) : 0, 7382521
benchmarks/2.9.0 (9.645M) : 0, 9644839
section Automatic
This PR (6000) (6.558M) : 0, 6558000
section Manual + Automatic
This PR (6000) (6.097M) : 0, 6096880
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6000) (10.148M) : 0, 10148468
master (10.229M) : 0, 10228633
benchmarks/2.9.0 (10.408M) : 0, 10408319
section Automatic
This PR (6000) (6.740M) : 0, 6739901
master (6.768M) : 0, 6768160
benchmarks/2.9.0 (7.412M) : 0, 7412035
section Trace stats
master (7.381M) : 0, 7380947
section Manual
master (10.100M) : 0, 10100381
section Manual + Automatic
This PR (6000) (6.203M) : 0, 6202889
master (6.294M) : 0, 6293834
section DD_TRACE_ENABLED=0
master (9.482M) : 0, 9482452
Summary of changes
Reason for change
The auto-pass code freeze check doesn't seem to be working for PRs to non-master for some reason.
Also the gitlab job that waits for artifacts to be ready before trying to download frequently times out.
Implementation details
Try to force the auto-check to work. It should already be working AFAICT based on the docs, so this is a stab in the dark 🤞
For the gitlab job, wait for longer