search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 26 Regressions on 10/11/2023 7:15:38 PM #23168

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare f20509b9ea563da18af963976b7db0e523e6837e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.49 ms 1.86 ms 1.25 0.03 False
893.37 μs 1.11 ms 1.24 0.03 False
150.08 μs 189.15 μs 1.26 0.09 False
1.11 ms 1.38 ms 1.24 0.03 False
374.40 μs 456.68 μs 1.22 0.04 False
299.79 μs 366.34 μs 1.22 0.06 False
757.51 μs 929.35 μs 1.23 0.04 False
222.22 μs 279.27 μs 1.26 0.07 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.GetMember.GetMethod20 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8634752083333332 > 1.5783978786057695. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -45.75652222010138 (T) = (0 -1862614.2900641025) / Math.Sqrt((198898459.35033417 / (18)) + (519702469.27226174 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.24226043052529495 = (1499375.0459205147 - 1862614.2900641025) / 1499375.0459205147 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod12 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1080440773809523 > 937.3552586805556. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -93.32385213425313 (T) = (0 -1110590.3111552633) / Math.Sqrt((24152606.559499774 / (18)) + (40314166.21728453 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.24190496240920933 = (894263.526414127 - 1110590.3111552633) / 894263.526414127 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod2 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 189.15423044217687 > 155.34750214460783. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -22.13111722413724 (T) = (0 -188346.53175743058) / Math.Sqrt((9712484.374687757 / (18)) + (23785268.238180403 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.2511302379217827 = (150541.10759107518 - 188346.53175743058) / 150541.10759107518 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod15 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3846970352564105 > 1.1734780078125. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -50.1818834896422 (T) = (0 -1387325.630437271) / Math.Sqrt((73849028.94017173 / (18)) + (240430609.74626148 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.23747078237538768 = (1121097.6858574627 - 1387325.630437271) / 1121097.6858574627 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod5 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 456.67747619047617 > 390.2211707848837. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -9.177957001963025 (T) = (0 -469262.55846755364) / Math.Sqrt((3369553.433289204 / (18)) + (1144953224.8629334 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.26494086601701167 = (370975.8859678131 - 469262.55846755364) / 370975.8859678131 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod4 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 366.3445436507936 > 312.76912625953156. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -37.32344674826278 (T) = (0 -370770.15435948904) / Math.Sqrt((16041554.697403863 / (18)) + (28675263.502739202 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.24248740043084843 = (298409.58888671204 - 370770.15435948904) / 298409.58888671204 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod10 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 929.3511519607844 > 783.0731421130952. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -57.68081289913167 (T) = (0 -927337.9640298426) / Math.Sqrt((132832394.43844543 / (18)) + (21622806.836975057 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.23783283275657116 = (749162.519760219 - 927337.9640298426) / 749162.519760219 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod3 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 279.2689400584795 > 233.77467371989553. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -41.21373336819123 (T) = (0 -279602.83420373395) / Math.Sqrt((7615760.245331648 / (18)) + (13820485.224607954 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.24694759015465118 = (224229.82041214462 - 279602.83420373395) / 224229.82041214462 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare f20509b9ea563da18af963976b7db0e523e6837e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
31.24 μs 62.38 μs 2.00 0.03 False
31.20 μs 62.62 μs 2.01 0.03 False
31.20 μs 62.37 μs 2.00 0.03 False
31.21 μs 62.37 μs 2.00 0.03 False
31.20 μs 62.58 μs 2.01 0.03 False
600.62 μs 1.29 ms 2.15 0.34 False
31.20 μs 62.37 μs 2.00 0.03 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 62.37568559096945 > 32.7892220247006. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -2665.6312023121573 (T) = (0 -62380.73190343023) / Math.Sqrt((2332.0173061548985 / (18)) + (68.72401730243965 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.9964763049425407 = (31245.41560999171 - 62380.73190343023) / 31245.41560999171 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 62.61791730769231 > 32.76457130210733. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -877.2311359729711 (T) = (0 -62440.2399045576) / Math.Sqrt((1712.4211490555201 / (18)) + (11712.307739420547 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.9998110692544091 = (31223.069451174324 - 62440.2399045576) / 31223.069451174324 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 62.37287101593626 > 32.764893338323354. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -1364.1819606361107 (T) = (0 -62394.622418494524) / Math.Sqrt((1674.8161038738203 / (18)) + (4291.271433743753 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.9984458549033468 = (31221.572636258483 - 62394.622418494524) / 31221.572636258483 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckArrayIsInterfaceYes #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 62.37307117683113 > 32.77947417664671. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -1377.4774866217135 (T) = (0 -62395.8225958109) / Math.Sqrt((1233.2614279267962 / (18)) + (4433.911041656193 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.9979489901747144 = (31229.937752492162 - 62395.8225958109) / 31229.937752492162 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckObjIsInterfaceYes #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 62.58325666666668 > 32.7634636976048. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 4:12:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -943.3737651024439 (T) = (0 -62436.78975056915) / Math.Sqrt((1170.9600788300104 / (18)) + (10302.252187772967 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -1.0001635838215497 = (31215.841671947783 - 62436.78975056915) / 31215.841671947783 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.FooObjIsDescendant #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2888401909722222 > 620.8770844907408. IsChangePoint: Marked as a change because one of 10/12/2023 7:03:34 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -2.5948229990707823 (T) = (0 -939960.2070655866) / Math.Sqrt((8338264203.371626 / (21)) + (110834451958.9174 / (7))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (7) - 2, .025) and -0.5424814329118627 = (609381.8615963177 - 939960.2070655866) / 609381.8615963177 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckObjIsInterfaceNo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 62.37424302788844 > 32.76513028030478. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -961.5674118830935 (T) = (0 -62419.30057109657) / Math.Sqrt((2044.7932251408715 / (18)) + (9385.82723187531 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.998786903961442 = (31228.59192612595 - 62419.30057109657) / 31228.59192612595 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare f20509b9ea563da18af963976b7db0e523e6837e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.92 ns 10.30 ns 1.15 0.03 False
12.84 ns 13.99 ns 1.09 0.04 False
20.00 ns 22.86 ns 1.14 0.10 False
52.03 ns 56.44 ns 1.08 0.04 False
52.13 ns 62.05 ns 1.19 0.09 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.298631104256 > 9.345773848086305. IsChangePoint: Marked as a change because one of 8/19/2023 8:04:10 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -208.27929526052424 (T) = (0 -10.295581124310166) / Math.Sqrt((0.0005935172834381176 / (18)) + (0.00010984586842085156 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.15490248002155937 = (8.9146757431138 - 10.295581124310166) / 8.9146757431138 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.BinarySearch(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.98987172522654 > 13.480838045247529. IsChangePoint: Marked as a change because one of 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/14/2023 10:44:26 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -29.00683599825334 (T) = (0 -13.993655366581757) / Math.Sqrt((0.017281620772823757 / (13)) + (0.0001730636360369903 / (15))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (15) - 2, .025) and -0.08213897717266645 = (12.931477066969121 - 13.993655366581757) / 12.931477066969121 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.BinarySearch(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.856697122878202 > 21.002980792195338. IsChangePoint: Marked as a change because one of 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/14/2023 10:44:26 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/9/2023 8:09:24 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -25.200553928798445 (T) = (0 -22.730597065048492) / Math.Sqrt((0.08804911907263503 / (13)) + (0.0719228271073096 / (15))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (15) - 2, .025) and -0.13538444758039617 = (20.02017652566 - 22.730597065048492) / 20.02017652566 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 56.43683467648357 > 54.63748737715633. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 7/5/2023 7:07:19 PM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -18.842630600576022 (T) = (0 -56.43672396217072) / Math.Sqrt((0.8257693589679419 / (18)) + (5.38250202518862E-07 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.07701873483735223 = (52.40087487493298 - 56.43672396217072) / 52.40087487493298 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 62.048815370221185 > 58.28544302819005. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 7/5/2023 7:07:19 PM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -20.45216322666609 (T) = (0 -62.2596830121038) / Math.Sqrt((2.8661213140597144 / (18)) + (0.06460894738920189 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.1543545865757936 = (53.93462609854316 - 62.2596830121038) / 53.93462609854316 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare f20509b9ea563da18af963976b7db0e523e6837e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.27 ms 3.58 ms 1.09 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.579958482142857 > 3.4354972750000004. IsChangePoint: Marked as a change because one of 6/26/2023 10:42:28 PM, 7/17/2023 6:26:12 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -221.73694092071665 (T) = (0 -3577901.8978365385) / Math.Sqrt((29593248.89560658 / (18)) + (2358329.2631834727 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.09286306470623276 = (3273879.4212962966 - 3577901.8978365385) / 3273879.4212962966 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare f20509b9ea563da18af963976b7db0e523e6837e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
24.55 μs 28.93 μs 1.18 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsFalse<Int32>.Span(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.93239168247945 > 25.700477456004315. IsChangePoint: Marked as a change because one of 7/17/2023 6:26:12 PM, 7/23/2023 4:13:29 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -24.335802307200893 (T) = (0 -28804.046466367894) / Math.Sqrt((2581.671189915036 / (18)) + (312542.25251078967 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.17606454329348073 = (24491.892584147055 - 28804.046466367894) / 24491.892584147055 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare f20509b9ea563da18af963976b7db0e523e6837e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
19.11 μs 23.14 μs 1.21 0.15 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.138209876543204 > 18.314410602616363. IsChangePoint: Marked as a change because one of 8/23/2023 5:32:05 AM, 9/18/2023 7:10:55 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -9.03539837594114 (T) = (0 -23159.05562742195) / Math.Sqrt((4755435.90527998 / (18)) + (1139.9336345705321 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.2509007077523219 = (18513.90400844464 - 23159.05562742195) / 18513.90400844464 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare f20509b9ea563da18af963976b7db0e523e6837e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
14.59 ns 16.94 ns 1.16 0.11 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_String.Trim(s: " Te st ") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.944358822327388 > 15.33183473652807. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -9.598473736715361 (T) = (0 -16.305945097873053) / Math.Sqrt((0.10231255653048953 / (18)) + (0.19782455774381164 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.10367707713731589 = (14.774199297648655 - 16.305945097873053) / 14.774199297648655 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare f20509b9ea563da18af963976b7db0e523e6837e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
25.68 μs 27.86 μs 1.09 0.15 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.863690476190477 > 26.96375327868853. IsChangePoint: Marked as a change because one of 6/25/2023 1:37:48 PM, 7/26/2023 10:32:59 PM, 9/19/2023 3:42:14 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -13.804512755075518 (T) = (0 -27890.768728409654) / Math.Sqrt((236194.49709066018 / (18)) + (73579.36315800872 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.07623060215750949 = (25915.23477635489 - 27890.768728409654) / 25915.23477635489 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare f20509b9ea563da18af963976b7db0e523e6837e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.40 ns 6.93 ns 1.28 0.06 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'
### Payloads [Baseline]() [Compare]() ### System.Text.Perf_Ascii.Equals_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.925172852493338 > 5.64882146651648. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/10/2023 8:47:51 AM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -174.00812620477097 (T) = (0 -6.944821127649935) / Math.Sqrt((4.1046433737780053E-05 / (18)) + (0.0007827597172704455 / (10))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (10) - 2, .025) and -0.29012925536783957 = (5.383042899581282 - 6.944821127649935) / 5.383042899581282 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)