search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x86: 48 Improvements on 8/27/2023 11:44:28 PM #21275

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
19.66 ns 14.01 ns 0.71 0.04 False
18.90 ns 13.72 ns 0.73 0.05 False
19.05 ns 14.55 ns 0.76 0.06 False
19.64 ns 14.24 ns 0.73 0.07 False
19.03 ns 14.45 ns 0.76 0.05 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, IgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.008171950198513 < 18.42555858874525. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 11.33841000991513 (T) = (0 -14.174529017834223) / Math.Sqrt((7.333174221142433 / (299)) + (0.006861462982269521 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11190053643448525 = (15.960519738326104 - 14.174529017834223) / 15.960519738326104 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, None, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.72090484827816 < 18.081394618220166. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 10.072205133298066 (T) = (0 -13.80052949373993) / Math.Sqrt((7.609964174648773 / (299)) + (0.07443382512272312 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10978613072471437 = (15.502487626905662 - 13.80052949373993) / 15.502487626905662 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, None, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.547827824874094 < 18.08510595080084. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 8.963092671263706 (T) = (0 -13.909268756245304) / Math.Sqrt((7.792430593791646 / (299)) + (0.19576090282820252 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1065061434964022 = (15.567279679655297 - 13.909268756245304) / 15.567279679655297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.239943397507334 < 18.439413149617465. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 9.911568550013113 (T) = (0 -14.311366788037189) / Math.Sqrt((7.530483615284082 / (299)) + (0.0787952223091898 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10462814366324573 = (15.983713009016675 - 14.311366788037189) / 15.983713009016675 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreNonSpace, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.4489271759972 < 18.06970025089135. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 7.676200393859132 (T) = (0 -13.809148845271912) / Math.Sqrt((6.617736422268687 / (299)) + (0.09814498016768135 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08258137400271978 = (15.052178421013277 - 13.809148845271912) / 15.052178421013277 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
407.71 ΞΌs 343.01 ΞΌs 0.84 0.13 False
748.38 ΞΌs 686.17 ΞΌs 0.92 0.05 False
187.13 ΞΌs 155.93 ΞΌs 0.83 0.03 False
187.11 ΞΌs 156.97 ΞΌs 0.84 0.04 False
2.61 ms 1.97 ms 0.76 0.05 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 343.01007375776396 < 387.6632611511752. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 18.225035303251847 (T) = (0 -343206.1772049565) / Math.Sqrt((1049187699.5755173 / (299)) + (193586.51368051703 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09056768149411305 = (377385.06782870047 - 343206.1772049565) / 377385.06782870047 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodSwitchCallType #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 686.1670833333334 < 710.9811403769841. IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 6/5/2023 10:43:22 AM, 6/13/2023 11:57:38 AM, 7/1/2023 6:36:20 PM, 8/28/2023 7:02:26 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 23.85055887857659 (T) = (0 -686409.4202851153) / Math.Sqrt((1509760059.8867934 / (299)) + (512715.66160439386 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.07258629681669054 = (740132.9287340085 - 686409.4202851153) / 740132.9287340085 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.ClassVirtualMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 155.93096269448372 < 177.7611953390731. IsChangePoint: Marked as a change because one of 5/12/2023 4:02:48 AM, 6/5/2023 10:43:22 AM, 6/23/2023 6:04:39 AM, 6/28/2023 3:57:02 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 20.58259271167563 (T) = (0 -156088.70924686023) / Math.Sqrt((235833737.2949762 / (299)) + (63459.54216770946 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.10500484394357587 = (174401.73635645886 - 156088.70924686023) / 174401.73635645886 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.SealedClassInterfaceMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.9655 < 177.75192607355442. IsChangePoint: Marked as a change because one of 5/12/2023 4:02:48 AM, 6/6/2023 4:05:41 PM, 6/23/2023 6:04:39 AM, 6/28/2023 3:57:02 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 21.25101248007262 (T) = (0 -156968.774739998) / Math.Sqrt((219927021.7604082 / (299)) + (24232.089140448184 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.10410081555307485 = (175208.0786153424 - 156968.774739998) / 175208.0786153424 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.EmptyStaticFunction #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.97277671875 < 2.479912082799145. IsChangePoint: Marked as a change because one of 6/5/2023 10:43:22 AM, 6/26/2023 10:42:28 PM, 7/14/2023 2:26:40 PM, 8/28/2023 1:23:07 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 22.748078868647333 (T) = (0 -1972251.5745972781) / Math.Sqrt((165887575052.17828 / (299)) + (5260139.972940688 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.21367332890679386 = (2508183.4905273104 - 1972251.5745972781) / 2508183.4905273104 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.69 ns 13.80 ns 0.88 0.10 False
21.50 ns 20.01 ns 0.93 0.12 False
20.76 ns 17.20 ns 0.83 0.16 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Int32.ToStringHex(value: 12345) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.800866279770602 < 14.904417686875716. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 16.776455575426287 (T) = (0 -13.57839597797645) / Math.Sqrt((1.5089419366542225 / (299)) + (0.024711319334129237 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0878490142815496 = (14.886127615464348 - 13.57839597797645) / 14.886127615464348 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.ToStringHex(value: 2147483647) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.014749756889326 < 22.93101103587248. IsChangePoint: Marked as a change because one of 5/24/2023 6:07:01 PM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 8/28/2023 1:23:07 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 17.520866672139338 (T) = (0 -20.06553957589766) / Math.Sqrt((3.1851604979234382 / (299)) + (0.02902034313913152 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.08720805115245144 = (21.982599212483784 - 20.06553957589766) / 21.982599212483784 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.ToStringHex(value: -2147483648) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.20272658318655 < 19.86593041676746. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 20.90992176073576 (T) = (0 -17.216366158233033) / Math.Sqrt((3.1174350265903596 / (299)) + (0.006696269645099356 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.11163492784556409 = (19.37983234356617 - 17.216366158233033) / 19.37983234356617 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
583.38 ΞΌs 439.45 ΞΌs 0.75 0.08 False
394.88 ΞΌs 312.50 ΞΌs 0.79 0.08 False
582.06 ΞΌs 438.68 ΞΌs 0.75 0.05 False
474.29 ΞΌs 407.26 ΞΌs 0.86 0.06 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsDescendantOfIFoo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 439.4529265873016 < 553.2240262345679. IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 112.8018930615352 (T) = (0 -438828.48880828376) / Math.Sqrt((430268795.6479186 / (299)) + (1192650.7698058737 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.23875501750990982 = (576461.5845123109 - 438828.48880828376) / 576461.5845123109 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 312.49599089635853 < 357.9776330959467. IsChangePoint: Marked as a change because one of 6/9/2023 2:21:24 AM, 7/5/2023 7:07:19 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 9.202896219188135 (T) = (0 -315504.2183085788) / Math.Sqrt((2857538806.8567133 / (299)) + (60016088.038125575 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0919709259757139 = (347460.4804340663 - 315504.2183085788) / 347460.4804340663 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFoo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 438.6800213675214 < 552.4483125413359. IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 119.27322621372917 (T) = (0 -438296.52230208553) / Math.Sqrt((387440874.84796077 / (299)) + (705745.7357617352 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.23854029603325325 = (575600.4159101584 - 438296.52230208553) / 575600.4159101584 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf2.CastingPerf.FooObjIsDescendant #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 407.25546016483514 < 454.9602378810816. IsChangePoint: Marked as a change because one of 6/9/2023 2:21:24 AM, 7/5/2023 7:07:19 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 18.921782016421073 (T) = (0 -418662.4080583865) / Math.Sqrt((2403119494.995946 / (299)) + (409875324.9888226 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.18467158227703706 = (513489.28720971197 - 418662.4080583865) / 513489.28720971197 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
90.35 ns 80.57 ns 0.89 0.12 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Byte>.DivideBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 80.5740204259031 < 85.83321731416632. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/8/2023 8:20:44 PM, 6/22/2023 7:55:14 AM, 7/14/2023 2:57:02 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 17.696233436098158 (T) = (0 -80.32058518324817) / Math.Sqrt((19.805694288652255 / (299)) + (0.2495139195318786 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0574855829103238 = (85.21947646303856 - 80.32058518324817) / 85.21947646303856 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.69 ns 15.62 ns 0.88 0.17 False
13.39 ns 11.23 ns 0.84 0.21 False
19.90 ns 17.51 ns 0.88 0.11 False
8.07 ns 6.74 ns 0.83 0.07 False
13.01 ns 10.81 ns 0.83 0.16 False
12.60 ns 10.87 ns 0.86 0.12 False
12.52 ns 9.90 ns 0.79 0.14 False
20.20 ns 17.28 ns 0.86 0.06 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_String.TrimStart_CharArr(s: " Test", c: [' ', 'β€…']) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.620766999629296 < 16.75693710105683. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 5.213653173482047 (T) = (0 -15.25932874082998) / Math.Sqrt((5.78149120975929 / (299)) + (0.47647318935952 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06335277303167729 = (16.291436414348183 - 15.25932874082998) / 16.291436414348183 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Trim(s: "Test ") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.22657336874804 < 12.741010713421808. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/23/2023 6:04:39 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 19.10129037404799 (T) = (0 -11.409416804594597) / Math.Sqrt((0.4315615455638337 / (299)) + (0.14903329105532243 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1277492754552849 = (13.080432590697956 - 11.409416804594597) / 13.080432590697956 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', 'β€…']) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.514104734238686 < 18.907639139546372. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 8.266699637803377 (T) = (0 -17.103054243987042) / Math.Sqrt((6.201588162091862 / (299)) + (0.07176498814899437 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06929937303919505 = (18.37653671711485 - 17.103054243987042) / 18.37653671711485 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.CtorCharCount(size: 1) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.7353752473046695 < 7.673172109143139. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 8/28/2023 1:23:07 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 21.73508242296109 (T) = (0 -6.8093188576059624) / Math.Sqrt((0.3621496780221567 / (299)) + (0.02522114325359986 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.13418915298838116 = (7.864672614242016 - 6.8093188576059624) / 7.864672614242016 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Trim(s: " Test") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.8110324664503 < 12.599063882841705. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 11.89930834788793 (T) = (0 -10.99247034433) / Math.Sqrt((1.5753564796372574 / (299)) + (0.16749681139742045 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10698303074631937 = (12.309363341120736 - 10.99247034433) / 12.309363341120736 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.TrimStart(s: " Test") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.87075456845029 < 12.109629200919413. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 9.269992417753745 (T) = (0 -10.992392526941181) / Math.Sqrt((2.053129701541901 / (299)) + (0.011365696740855087 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0673822516868783 = (11.786600187294034 - 10.992392526941181) / 11.786600187294034 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.TrimEnd(s: "Test ") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.898289134353481 < 12.475594945776773. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 15.063358349781046 (T) = (0 -10.350922179012265) / Math.Sqrt((2.5121291685340816 / (299)) + (0.11923042142794261 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.14403262415904253 = (12.092659686758333 - 10.350922179012265) / 12.092659686758333 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', 'β€…']) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.28318252832295 < 19.1233088038477. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 5.152044514019623 (T) = (0 -17.497465846101193) / Math.Sqrt((6.990414830613583 / (299)) + (0.32028924474957055 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05341266695751441 = (18.484787652779477 - 17.497465846101193) / 18.484787652779477 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.98 ns 6.79 ns 0.85 0.04 False
7.61 ns 6.43 ns 0.85 0.02 False
7.98 ns 6.80 ns 0.85 0.05 False
7.59 ns 6.46 ns 0.85 0.03 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Boolean*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Boolean.TryParse(value: "False") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.7906008932196515 < 7.5792361496421385. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 12.277313770479736 (T) = (0 -6.80379055926428) / Math.Sqrt((0.36933632203945876 / (299)) + (0.00019866757799529561 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0598255388351561 = (7.236731947424541 - 6.80379055926428) / 7.236731947424541 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Boolean.Parse(value: "false") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.433476938014854 < 7.211203050680386. IsChangePoint: Marked as a change because one of 5/17/2023 4:23:45 PM, 6/22/2023 7:55:14 AM, 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 19.59783239767796 (T) = (0 -6.435717686221263) / Math.Sqrt((0.27431816008083393 / (299)) + (5.8839616857992E-05 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08455057962990449 = (7.030118259968364 - 6.435717686221263) / 7.030118259968364 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Boolean.TryParse(value: "false") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.802155089214819 < 7.578990644992933. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 12.176841335508763 (T) = (0 -6.803527561281242) / Math.Sqrt((0.367670740248171 / (299)) + (0.00014529710379448707 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.059191857459939635 = (7.231578101474119 - 6.803527561281242) / 7.231578101474119 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Boolean.Parse(value: "False") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.461735238402753 < 7.2105345441096045. IsChangePoint: Marked as a change because one of 5/17/2023 4:23:45 PM, 6/22/2023 7:55:14 AM, 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 19.694074075498783 (T) = (0 -6.43549844887056) / Math.Sqrt((0.2739097867786297 / (299)) + (5.934281404448977E-05 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08487634226165569 = (7.032381246459506 - 6.43549844887056) / 7.032381246459506 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.34 ms 2.78 ms 0.83 0.10 False
105.12 ΞΌs 95.21 ΞΌs 0.91 0.02 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.775885977564102 < 3.024907275545635. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 7/20/2023 8:15:02 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 12.390429534291684 (T) = (0 -2814418.439797327) / Math.Sqrt((36567747039.16372 / (299)) + (13266381706.263168 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10264443159417473 = (3136346.994310418 - 2814418.439797327) / 3136346.994310418 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: Compiled) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 95.21288087934559 < 99.4353506905418. IsChangePoint: Marked as a change because one of 5/21/2023 8:41:51 PM, 8/28/2023 1:23:07 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 35.38190938266228 (T) = (0 -95812.23059969436) / Math.Sqrt((3784751.4613349703 / (299)) + (833142.0099214159 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.07664691870302037 = (103765.53946742948 - 95812.23059969436) / 103765.53946742948 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
253.69 ns 205.81 ns 0.81 0.13 False
1.06 ΞΌs 889.87 ns 0.84 0.04 False
714.54 ns 611.31 ns 0.86 0.20 False
977.00 ns 867.72 ns 0.89 0.07 False
953.24 ns 822.71 ns 0.86 0.08 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_0 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 205.8089760938205 < 241.0339785845573. IsChangePoint: Marked as a change because one of 7/28/2023 3:39:20 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 4.416898566657095 (T) = (0 -213.40578855063578) / Math.Sqrt((661.5548166131812 / (94)) + (123.0715903392987 / (24))) is greater than 1.9806260024587088 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (94) + (24) - 2, .975) and 0.06733013913949303 = (228.81171302538039 - 213.40578855063578) / 228.81171302538039 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_5 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 889.8704310658769 < 0.9985881352462379. IsChangePoint: Marked as a change because one of 7/28/2023 3:39:20 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 6.434671974599548 (T) = (0 -901.7507536155321) / Math.Sqrt((8421.245529102687 / (93)) + (881.6346843004001 / (24))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (93) + (24) - 2, .975) and 0.07450794436570503 = (974.3473735141988 - 901.7507536155321) / 974.3473735141988 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_1 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 611.308710706201 < 673.8217997265923. IsChangePoint: Marked as a change because one of 7/28/2023 3:39:20 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 5.860969373700179 (T) = (0 -596.4306386919014) / Math.Sqrt((5124.0885798885365 / (95)) + (667.4503302769783 / (24))) is greater than 1.9804475986831467 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (95) + (24) - 2, .975) and 0.08159833236106577 = (649.4224256204047 - 596.4306386919014) / 649.4224256204047 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_4 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 867.7175055335454 < 0.9595154150928314. IsChangePoint: Marked as a change because one of 7/28/2023 3:39:20 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 6.611143776839288 (T) = (0 -856.2073894664194) / Math.Sqrt((8913.402160596808 / (96)) + (987.705902469104 / (24))) is greater than 1.9802722492727756 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (96) + (24) - 2, .975) and 0.08204900826420275 = (932.7375831332521 - 856.2073894664194) / 932.7375831332521 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance_2 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 822.7116004012885 < 906.8021283431626. IsChangePoint: Marked as a change because one of 7/28/2023 3:39:20 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 5.600373970125063 (T) = (0 -813.0107508966117) / Math.Sqrt((9390.670647143588 / (93)) + (1205.0204535071812 / (24))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (93) + (24) - 2, .975) and 0.07808452287361997 = (881.8712464083742 - 813.0107508966117) / 881.8712464083742 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
234.19 ns 209.27 ns 0.89 0.02 False
171.54 ΞΌs 147.29 ΞΌs 0.86 0.01 False
183.41 ns 163.08 ns 0.89 0.02 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
### Payloads [Baseline]() [Compare]() ### System.Text.Tests.Perf_StringBuilder.Append_Char(length: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 209.26775984314799 < 222.6756384955198. IsChangePoint: Marked as a change because one of 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 7/7/2023 11:40:29 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 24.793869492387824 (T) = (0 -209.1908283377336) / Math.Sqrt((195.57303142905343 / (299)) + (0.6455972489953434 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08909348360235653 = (229.65125901724724 - 209.1908283377336) / 229.65125901724724 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_StringBuilder.Append_Char(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 147.289579248366 < 162.5032813726452. IsChangePoint: Marked as a change because one of 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 7/7/2023 11:40:29 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 17.70563728425406 (T) = (0 -148039.71046497906) / Math.Sqrt((119830108.1711516 / (299)) + (2961162.2121790648 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07968864992242572 = (160858.29046061487 - 148039.71046497906) / 160858.29046061487 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 163.0812915433607 < 172.25390072471478. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 7/7/2023 11:40:29 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 14.83748048683387 (T) = (0 -163.38350591756063) / Math.Sqrt((96.81096089031675 / (299)) + (1.2351660086804366 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05269860999795511 = (172.47257065378943 - 163.38350591756063) / 172.47257065378943 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_Timer

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.92 secs 3.30 secs 0.67 0.39 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Timer*'
### Payloads [Baseline]() [Compare]() ### System.Threading.Tests.Perf_Timer.SynchronousContention #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.30123882 < 4.6729563205. IsChangePoint: Marked as a change because one of 6/21/2023 9:03:14 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 8.323330942048448 (T) = (0 -3482541034.4658113) / Math.Sqrt((8.688545367887619E+17 / (299)) + (89644910603363020 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1630181862510187 = (4160832382.805223 - 3482541034.4658113) / 4160832382.805223 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
488.86 ns 402.50 ns 0.82 0.08 False
10.66 ΞΌs 9.76 ΞΌs 0.92 0.07 False
678.08 ns 616.41 ns 0.91 0.05 False
377.79 ns 284.30 ns 0.75 0.07 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### System.Linq.Tests.Perf_Enumerable.Range #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 402.4988764724693 < 464.4344928828712. IsChangePoint: Marked as a change because one of 6/2/2023 6:53:36 PM, 8/10/2023 8:47:51 AM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 22.708221637522964 (T) = (0 -403.26655512936117) / Math.Sqrt((981.0150887497563 / (299)) + (33.67086477645059 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10863106948687497 = (452.41262211957167 - 403.26655512936117) / 452.41262211957167 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.763351816645542 < 10.097827426877679. IsChangePoint: Marked as a change because one of 6/26/2023 10:42:28 PM, 8/28/2023 1:23:07 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 36.49805166531108 (T) = (0 -9710.01282575328) / Math.Sqrt((90492.73742534881 / (299)) + (9468.133576647308 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09236729475211597 = (10698.17423899613 - 9710.01282575328) / 10698.17423899613 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 616.4093982454054 < 644.161068379278. IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 7/1/2023 6:36:20 PM, 7/5/2023 7:07:19 PM, 7/17/2023 6:26:12 PM, 7/21/2023 12:22:03 PM, 8/28/2023 1:23:07 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 16.739901159479466 (T) = (0 -614.8073079314825) / Math.Sqrt((1213.0861784269973 / (299)) + (0.6239479004636855 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.052163896506541865 = (648.6430572389837 - 614.8073079314825) / 648.6430572389837 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 284.30490778454435 < 358.1644963901263. IsChangePoint: Marked as a change because one of 6/16/2023 2:41:30 PM, 6/22/2023 7:55:14 AM, 8/22/2023 10:44:27 PM, 8/28/2023 1:23:07 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 10.027249672054166 (T) = (0 -284.40162783429474) / Math.Sqrt((1119.147074095864 / (299)) + (0.08038886228929049 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.06388499014290012 = (303.81056263344107 - 284.40162783429474) / 303.81056263344107 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.01 ns 12.76 ns 0.71 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.762448119504693 < 16.935913314917826. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 27.85781336966969 (T) = (0 -12.55956733329411) / Math.Sqrt((8.414978594763811 / (299)) + (0.006390606720390229 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.27212338057955493 = (17.25507730045564 - 12.55956733329411) / 17.25507730045564 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.12 ΞΌs 13.95 ΞΌs 0.92 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsKeyTrue<String, String>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.947939199172577 < 14.358592836564071. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 7/14/2023 2:26:40 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 13.788322189854542 (T) = (0 -13826.794303734443) / Math.Sqrt((698480.2606876409 / (299)) + (21916.924286368714 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05378626316682036 = (14612.760062024072 - 13826.794303734443) / 14612.760062024072 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 538cd03b39afb57eb17ea77a86b3c94c039b8822
Compare c7d36b48a493789c9267069645e2690e804a2067
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
180.55 ns 160.42 ns 0.89 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList*'
### Payloads [Baseline]() [Compare]() ### XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.Enumerator #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 160.41815162095435 < 171.6326430104014. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/16/2023 8:27:26 PM, 8/27/2023 2:42:29 PM, 9/4/2023 7:24:21 PM falls between 8/27/2023 6:19:41 AM and 9/4/2023 7:24:21 PM. IsImprovementStdDev: Marked as improvement because 23.264220571053393 (T) = (0 -159.03535116036588) / Math.Sqrt((93.55170070193773 / (299)) + (1.8989481234355758 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08390379425883589 = (173.60114599721427 - 159.03535116036588) / 173.60114599721427 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)