search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 17 Improvements on 9/19/2023 3:42:14 PM #22158

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
131.27 μs 109.42 μs 0.83 0.03 False
302.71 μs 262.94 μs 0.87 0.04 False
131.38 μs 109.51 μs 0.83 0.06 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.LowLevelPerf.ClassVirtualMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 109.42211879706832 < 124.71125103650837. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 9.590154505083667 (T) = (0 -109419.77572889408) / Math.Sqrt((117220019.4561587 / (131)) + (42024.452058671886 / (30))) is greater than 1.9749962127672496 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (30) - 2, .975) and 0.07661560531973109 = (118498.61916583696 - 109419.77572889408) / 118498.61916583696 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 262.9429718005952 < 288.5229604095123. IsChangePoint: Marked as a change because one of 8/15/2023 7:48:35 PM, 8/28/2023 1:23:07 PM, 9/15/2023 8:45:09 AM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 9.765504081689325 (T) = (0 -262838.49676449125) / Math.Sqrt((398479782.65323913 / (132)) + (71717.363925282 / (29))) is greater than 1.9749962127672496 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (29) - 2, .975) and 0.06066263691489149 = (279812.67124437465 - 262838.49676449125) / 279812.67124437465 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.GenericGenericMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 109.51314613511491 < 124.85594930705935. IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 8:56:48 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 8.97868203507626 (T) = (0 -109471.34879975328) / Math.Sqrt((114669819.24198441 / (131)) + (21040.389006128866 / (29))) is greater than 1.9750920727118966 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (29) - 2, .975) and 0.07129494789337469 = (117875.25926712068 - 109471.34879975328) / 117875.25926712068 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
361.15 ns 318.13 ns 0.88 0.36 False
602.56 ns 316.60 ns 0.53 0.34 False
409.75 ns 309.64 ns 0.76 0.39 False
361.71 ns 316.81 ns 0.88 0.52 False
546.10 ns 420.46 ns 0.77 0.23 False
548.82 ns 491.24 ns 0.90 0.59 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 318.13298936835514 < 391.0890395013886. IsChangePoint: Marked as a change because one of 9/18/2023 8:11:22 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 7.694629879301397 (T) = (0 -329.0688823500334) / Math.Sqrt((4267.556905921401 / (131)) + (1009.4845358925252 / (30))) is greater than 1.9749962127672496 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (30) - 2, .975) and 0.15986861896690333 = (391.68740720693285 - 329.0688823500334) / 391.68740720693285 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 316.595543291411 < 481.8429586582162. IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/1/2023 9:23:50 AM, 9/16/2023 11:57:54 AM, 9/17/2023 8:50:51 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 4.1711157440378415 (T) = (0 -347.0251965959887) / Math.Sqrt((5929.755323882263 / (124)) + (3428.6442255955144 / (38))) is greater than 1.9749015600005977 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (124) + (38) - 2, .975) and 0.12374731647520933 = (396.0332483092142 - 347.0251965959887) / 396.0332483092142 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 309.6405846607846 < 388.2118215500891. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 10.794787512393807 (T) = (0 -317.13903011733436) / Math.Sqrt((1697.0609740850173 / (131)) + (627.3300546665199 / (30))) is greater than 1.9749962127672496 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (30) - 2, .975) and 0.1653323359524597 = (379.9584478682656 - 317.13903011733436) / 379.9584478682656 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 316.81258063310156 < 367.19670753042755. IsChangePoint: Marked as a change because one of 9/15/2023 4:40:29 AM, 9/16/2023 1:27:38 AM, 9/19/2023 8:41:15 PM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 7.831305919332686 (T) = (0 -330.5340550489613) / Math.Sqrt((5455.3174919002 / (133)) + (1115.6722704052697 / (27))) is greater than 1.9750920727118966 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (133) + (27) - 2, .975) and 0.17694846475398734 = (401.5958186022503 - 330.5340550489613) / 401.5958186022503 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 420.46357337543185 < 491.91944526595614. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 8.314433365684726 (T) = (0 -434.3642218554562) / Math.Sqrt((524.6615179149644 / (133)) + (1772.766193913005 / (29))) is greater than 1.9749015600005977 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (133) + (29) - 2, .975) and 0.13375910536248153 = (501.4358298533313 - 434.3642218554562) / 501.4358298533313 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 491.237268321513 < 520.2432844318049. IsChangePoint: Marked as a change because one of 9/12/2023 4:19:08 AM, 9/19/2023 5:34:53 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 3.474777423304906 (T) = (0 -496.22352300901986) / Math.Sqrt((5430.050574748511 / (132)) + (6794.746054521755 / (30))) is greater than 1.9749015600005977 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (132) + (30) - 2, .975) and 0.10278126955612235 = (553.0686176864866 - 496.22352300901986) / 553.0686176864866 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
20.29 ns 16.97 ns 0.84 0.14 False
27.95 ns 24.68 ns 0.88 0.27 False
27.83 ns 21.32 ns 0.77 0.43 False
250.77 ns 187.42 ns 0.75 0.12 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.972389565889845 < 19.08052819957604. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 20.80700661815516 (T) = (0 -17.079358444908532) / Math.Sqrt((0.9599158521003918 / (131)) + (0.09026585837309477 / (31))) is greater than 1.9749015600005977 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (31) - 2, .975) and 0.10974613968360734 = (19.18481818078115 - 17.079358444908532) / 19.18481818078115 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.675261278358956 < 26.56781839875563. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 22.429969651514597 (T) = (0 -24.671169997857326) / Math.Sqrt((4.603151419132931 / (131)) + (0.0018313944709832236 / (29))) is greater than 1.9750920727118966 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (29) - 2, .975) and 0.14572054725197228 = (28.87950765817396 - 24.671169997857326) / 28.87950765817396 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.316477179485727 < 24.5403434368633. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 20.190663479455 (T) = (0 -21.243045735306286) / Math.Sqrt((3.623814735520535 / (131)) + (0.01927252768923523 / (29))) is greater than 1.9750920727118966 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (29) - 2, .975) and 0.13790810458723682 = (24.641277627526325 - 21.243045735306286) / 24.641277627526325 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 187.41749700386833 < 237.79627182290244. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 45.395836570591335 (T) = (0 -188.38450510354022) / Math.Sqrt((181.53116643205774 / (131)) + (0.24038114734908786 / (29))) is greater than 1.9750920727118966 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (29) - 2, .975) and 0.2214963141054083 = (241.9828043422356 - 188.38450510354022) / 241.9828043422356 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.90 μs 6.94 μs 0.88 0.07 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: DeepTree) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.9358312943066025 < 7.491021625421356. IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 25.81873227863674 (T) = (0 -7016.420056946784) / Math.Sqrt((56567.49765414381 / (131)) + (3238.0924041946682 / (30))) is greater than 1.9749962127672496 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (30) - 2, .975) and 0.07875707387932832 = (7616.253930429331 - 7016.420056946784) / 7616.253930429331 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.TreeInsert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
54.15 μs 48.68 μs 0.90 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.TreeInsert*'
### Payloads [Baseline]() [Compare]() ### Benchstone.BenchI.TreeInsert.Test #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 48.67853912396481 < 51.62891542457119. IsChangePoint: Marked as a change because one of 8/8/2023 9:37:01 AM, 8/22/2023 4:48:44 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 13.540914537458196 (T) = (0 -48641.16663213956) / Math.Sqrt((6660376.23357308 / (131)) + (16701.918413427044 / (31))) is greater than 1.9749015600005977 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (31) - 2, .975) and 0.059356949230591344 = (51710.546941640634 - 48641.16663213956) / 51710.546941640634 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
321.40 μs 268.68 μs 0.84 0.44 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<String>.Queue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 268.6773849929378 < 297.5947104373937. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 8.874664705060226 (T) = (0 -274096.5289816595) / Math.Sqrt((177837278.86101806 / (131)) + (243033688.9862894 / (30))) is greater than 1.9749962127672496 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (131) + (30) - 2, .975) and 0.09056016132134853 = (301390.5014100783 - 274096.5289816595) / 301390.5014100783 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 963954a11c1beeea4ad63002084a213d8d742864
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.87 ns 3.46 ns 0.71 0.29 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.4632246140096603 < 4.625082405225074. IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/19/2023 8:08:41 AM, 9/25/2023 10:28:34 PM falls between 9/17/2023 8:50:51 AM and 9/25/2023 10:28:34 PM. IsImprovementStdDev: Marked as improvement because 4.85852564522144 (T) = (0 -3.4405057570546793) / Math.Sqrt((0.48858142435118157 / (133)) + (0.008043284298798964 / (30))) is greater than 1.9748080917515904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (133) + (30) - 2, .975) and 0.08143848864323801 = (3.7455365966432423 - 3.4405057570546793) / 3.7455365966432423 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)