Open kevingosse opened 1 month ago
Branch report: kevin/better_loader
Commit report: a1fdf2e
Test service: dd-trace-dotnet
:x: 118 Failed (8 Known Flaky), 349922 Passed, 1526 Skipped, 9h 27m 18.66s Total Time :hourglass: 1 Performance Regression
This report shows up to 5 failed tests.
SubmitsTraces
-Datadog.Trace.ClrProfiler.IntegrationTests.AWS.AwsLambdaTests
- Details
<details>
<summary>Expand for error</summary>
```
Expected relevantSpans to contain at least 76 item(s) because we want to ensure that we don't timeout while waiting for spans from the mock tracer agent, but found 0: {empty}.
```
</details>
SubmitsTraces
- Datadog.Trace.ClrProfiler.IntegrationTests.AWS.AwsLambdaTests
- Details
SubmitsTraces
- Datadog.Trace.ClrProfiler.IntegrationTests.AWS.AwsLambdaTests
- Details
DetectAgentUrl
- Datadog.Trace.Tools.dd_dotnet.ArtifactTests.Checks.AgentConnectivityCheckTests
- Details
DetectAgentUrl
- Datadog.Trace.Tools.dd_dotnet.ArtifactTests.Checks.AgentConnectivityCheckTests
- Details
Baseline_liveheap
- scenarios
2.97s (+218.43ms, +8%) - DetailsExecution-time results for samples comparing the following branches/commits:
Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.
Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).
gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5968) - mean (70ms) : 68, 72
. : milestone, 70,
master - mean (69ms) : 67, 72
. : milestone, 69,
section CallTarget+Inlining+NGEN
This PR (5968) - mean (1,099ms) : 1078, 1120
. : milestone, 1099,
master - mean (1,100ms) : 1075, 1126
. : milestone, 1100,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5968) - mean (109ms) : 106, 111
. : milestone, 109,
master - mean (108ms) : 106, 110
. : milestone, 108,
section CallTarget+Inlining+NGEN
This PR (5968) - mean (768ms) : 753, 784
. : milestone, 768,
master - mean (764ms) : 750, 779
. : milestone, 764,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5968) - mean (92ms) : 89, 94
. : milestone, 92,
master - mean (91ms) : 89, 94
. : milestone, 91,
section CallTarget+Inlining+NGEN
This PR (5968) - mean (728ms) : 712, 744
. : milestone, 728,
master - mean (726ms) : 709, 743
. : milestone, 726,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5968) - mean (190ms) : 187, 193
. : milestone, 190,
master - mean (190ms) : 186, 193
. : milestone, 190,
section CallTarget+Inlining+NGEN
This PR (5968) - mean (1,192ms) : 1173, 1212
. : milestone, 1192,
master - mean (1,192ms) : 1166, 1219
. : milestone, 1192,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5968) - mean (275ms) : 271, 279
. : milestone, 275,
master - mean (276ms) : 272, 280
. : milestone, 276,
section CallTarget+Inlining+NGEN
This PR (5968) - mean (938ms) : 918, 958
. : milestone, 938,
master - mean (939ms) : 922, 956
. : milestone, 939,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5968) - mean (265ms) : 260, 269
. : milestone, 265,
master - mean (264ms) : 261, 268
. : milestone, 264,
section CallTarget+Inlining+NGEN
This PR (5968) - mean (926ms) : 901, 951
. : milestone, 926,
master - mean (924ms) : 906, 942
. : milestone, 924,
Benchmarks for #5968 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored.
Summary of changes
Instead of loading the native trace/profiler as soon as the native loader is loaded, wait until we've checked the target runtime/process name.
Reason for change
When returning an error code from
ICorProfilerCallback::Initialize
, .NET unloads the native loader. However, the native tracer/profiler stay in memory (because .NET is not aware of them). Therefore it makes sense to perform as many checks as possible before actually loading them. Plus that's less work to do in the most common failure cases (blocklisted process).There are still cases where we're not unloading the native tracer/profiler even though we should (for instance, if their
Initialize
method failed). However they're trickier to get right (we must make sure there's no background thread running, for instance the logger) so I'll leave that for a future PR.Implementation details
Moved the
LoadConfiguration
/LoadClassFactory
/LoadInstance
methods of the dynamic dispatcher into a singleInitialize
method.Test coverage
Other details