search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.12k stars 4.7k forks source link

[Perf] Windows/x64: 20 Regressions on 7/12/2023 7:15:00 AM #89103

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare 82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.TryParseSpan(value%3a%20%2212345%22).html>)
📝 - Benchmark Source		 10.99 ns 14.19 ns 1.29 0.08 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.ToString(value%3a%202147483647).html>)
📝 - Benchmark Source		 10.90 ns 14.99 ns 1.37 0.13 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.TryParse(value%3a%20%22-2147483648%22).html>)
📝 - Benchmark Source		 15.58 ns 20.40 ns 1.31 0.11 False
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.TryParseSpan(value%3a%20%22-2147483648%22).html>)
📝 - Benchmark Source		 16.62 ns 19.16 ns 1.15 0.09 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.Parse(value%3a%20%222147483647%22).html>)
📝 - Benchmark Source		 14.40 ns 17.33 ns 1.20 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.ToString(value%3a%2012345).html>)
📝 - Benchmark Source		 8.53 ns 11.91 ns 1.40 0.12 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.ParseSpan(value%3a%20%2212345%22).html>)
📝 - Benchmark Source		 11.63 ns 16.67 ns 1.43 0.09 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.Parse(value%3a%20%2212345%22).html>)
📝 - Benchmark Source		 11.10 ns 16.83 ns 1.52 0.12 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.TryParse(value%3a%20%222147483647%22).html>)
📝 - Benchmark Source		 14.38 ns 19.78 ns 1.38 0.06 False
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int32.TryParseSpan(value%3a%20%222147483647%22).html>)
📝 - Benchmark Source		 14.30 ns 19.68 ns 1.38 0.08 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Int32.TryParseSpan(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.190120288945042 > 11.534074752675743. IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -24.697939964773763 (T) = (0 -14.407504759071363) / Math.Sqrt((0.000210871079473183 / (19)) + (0.1920708851182739 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.31172436769823975 = (10.98363735085067 - 14.407504759071363) / 10.98363735085067 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.ToString(value: 2147483647) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.993700539555588 > 12.368893105520494. IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 6/2/2023 6:53:36 PM, 6/5/2023 10:43:22 AM, 7/12/2023 9:48:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -27.260590010041422 (T) = (0 -15.496795506087762) / Math.Sqrt((0.18115007417294351 / (22)) + (0.08147438563616338 / (7))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (7) - 2, .025) and -0.3297635832937344 = (11.653797487597945 - 15.496795506087762) / 11.653797487597945 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.TryParse(value: "-2147483648") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.396802256987975 > 16.3567919600678. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -26.031347472653543 (T) = (0 -19.70150746050102) / Math.Sqrt((0.011863208373597493 / (19)) + (0.23923420674679818 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.2610591640305737 = (15.622984252008788 - 19.70150746050102) / 15.622984252008788 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.16131198673049 > 17.506325757575464. IsChangePoint: Marked as a change because one of 4/26/2023 6:27:29 AM, 4/26/2023 4:28:25 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -23.009844760978734 (T) = (0 -19.84466188881448) / Math.Sqrt((0.14091681560057817 / (19)) + (0.14438831076899844 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.20687769829617622 = (16.44297671324146 - 19.84466188881448) / 16.44297671324146 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.Parse(value: "2147483647") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.329416854116946 > 15.091691023660475. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -16.312934516703194 (T) = (0 -18.19734938679178) / Math.Sqrt((0.0015169596589323792 / (19)) + (0.548154216803538 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.2658786494521896 = (14.375271590737947 - 18.19734938679178) / 14.375271590737947 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.ToString(value: 12345) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.90771421298525 > 9.010497568013985. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/22/2023 7:55:14 AM, 7/12/2023 9:48:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -21.22220608159406 (T) = (0 -12.28222343184322) / Math.Sqrt((0.07844561528440665 / (22)) + (0.1741739647391373 / (7))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (7) - 2, .025) and -0.4112970353405608 = (8.702791208570343 - 12.28222343184322) / 8.702791208570343 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.ParseSpan(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.66542346740148 > 12.211811377879068. IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -25.408789976545123 (T) = (0 -15.937928289814147) / Math.Sqrt((1.1162861616165135E-05 / (19)) + (0.28747723161934796 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.3704405032137533 = (11.629784914003116 - 15.937928289814147) / 11.629784914003116 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.Parse(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.826361980205 > 11.649755632515522. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -16.09636804454305 (T) = (0 -15.181679741034051) / Math.Sqrt((0.0041244532777064075 / (19)) + (0.5743099533001782 / (9))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (9) - 2, .025) and -0.36665350673470265 = (11.108653119624378 - 15.181679741034051) / 11.108653119624378 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.TryParse(value: "2147483647") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.77535313009241 > 15.093973076160415. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -50.573425619394655 (T) = (0 -19.0281170471599) / Math.Sqrt((0.0034886385822024257 / (19)) + (0.08235326096061854 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.3225208125873535 = (14.387763781148871 - 19.0281170471599) / 14.387763781148871 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.TryParseSpan(value: "2147483647") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.680236497246323 > 14.97101717214047. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -71.27114399575764 (T) = (0 -19.312547634887263) / Math.Sqrt((0.0017930930556328644 / (19)) + (0.049113813075204606 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.3533653853715328 = (14.270017427396729 - 19.312547634887263) / 14.270017427396729 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare 82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[EndsWith - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Int32).EndsWith(Size%3a%20512).html>)
📝 - Benchmark Source		 19.98 ns 23.93 ns 1.20 0.11 False
[IndexOfAnyFiveValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Int32).IndexOfAnyFiveValues(Size%3a%2033).html>)
📝 - Benchmark Source		 32.27 ns 43.41 ns 1.35 0.18 False
[StartsWith - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Int32).StartsWith(Size%3a%20512).html>)
📝 - Benchmark Source		 17.61 ns 23.07 ns 1.31 0.17 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Int32>.EndsWith(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.928167560437533 > 20.999560282601973. IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/2/2023 6:53:36 PM, 6/8/2023 8:20:44 PM, 6/22/2023 7:55:14 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -8.54359133045494 (T) = (0 -22.311792180136468) / Math.Sqrt((0.22509452045072115 / (19)) + (0.6775623704422623 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.12112244848097427 = (19.901298212668074 - 22.311792180136468) / 19.901298212668074 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.41256488683741 > 33.98140889328459. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/13/2023 5:43:42 PM, 6/19/2023 6:37:24 PM, 7/5/2023 7:07:19 PM, 7/12/2023 9:48:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -59.5690257957098 (T) = (0 -42.79789455195748) / Math.Sqrt((0.1796580122748577 / (22)) + (0.15216707898017826 / (7))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (7) - 2, .025) and -0.31699264779164527 = (32.49668449077653 - 42.79789455195748) / 32.49668449077653 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.StartsWith(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.071253447888303 > 18.3165331355214. IsChangePoint: Marked as a change because one of 5/16/2023 5:40:18 PM, 6/22/2023 7:55:14 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -7.109646695955824 (T) = (0 -22.868928049019054) / Math.Sqrt((0.1943529829533172 / (19)) + (5.747053214192557 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.3119387828064188 = (17.43139874262978 - 22.868928049019054) / 17.43139874262978 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare 82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[GetSingle - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Get.GetSingle.html>)
📝 - Benchmark Source		 4.53 Ξs 4.82 Ξs 1.06 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Tests.Perf_Get.GetSingle #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.819585599387208 > 4.758078055162659. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -12.816471219622851 (T) = (0 -4857.4633481907085) / Math.Sqrt((1694.7576055906093 / (19)) + (4334.554670054745 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.06419276112964237 = (4564.458174884128 - 4857.4633481907085) / 4564.458174884128 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare 82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsFalse(Int32).List(Size%3a%20512).html>)
📝 - Benchmark Source		 16.44 Ξs 20.26 Ξs 1.23 0.10 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsFalse<Int32>.List(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.259371761658027 > 17.259348547581492. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/2/2023 6:53:36 PM, 6/13/2023 11:57:38 AM, 6/19/2023 11:51:49 PM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -202.4740960409093 (T) = (0 -20369.29188027515) / Math.Sqrt((94.10078871264959 / (19)) + (3726.473764407471 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.2393978850339125 = (16434.828658528655 - 20369.29188027515) / 16434.828658528655 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare 82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsTrue(Int32).Queue(Size%3a%20512).html>)
📝 - Benchmark Source		 13.20 Ξs 16.68 Ξs 1.26 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<Int32>.Queue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.682858635716826 > 13.861466273593738. IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/2/2023 6:53:36 PM, 6/14/2023 4:56:58 PM, 6/27/2023 10:31:09 PM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -2.232653896944129 (T) = (0 -15998.855719533469) / Math.Sqrt((3352948.64773084 / (19)) + (3292236.2439783234 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.110170783211233 = (14411.166247102861 - 15998.855719533469) / 14411.166247102861 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare 82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int64.ParseSpan(value%3a%20%2212345%22).html>)
📝 - Benchmark Source		 13.02 ns 15.32 ns 1.18 0.09 False
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int64.TryParseSpan(value%3a%20%2212345%22).html>)
📝 - Benchmark Source		 12.21 ns 15.12 ns 1.24 0.06 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Int64.ParseSpan(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.318294192831162 > 13.648609928158619. IsChangePoint: Marked as a change because one of 4/19/2023 8:38:27 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -12.295197589970563 (T) = (0 -15.048104182117617) / Math.Sqrt((0.005490628588379624 / (19)) + (0.269115252900472 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.15574090629500364 = (13.020309396470024 - 15.048104182117617) / 13.020309396470024 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int64.TryParseSpan(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.117739991647596 > 12.997921566357508. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -30.475514094506334 (T) = (0 -15.025658001356609) / Math.Sqrt((0.0063871462073339935 / (19)) + (0.07486717501870088 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.21860734658284697 = (12.33018826243806 - 15.025658001356609) / 12.33018826243806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare 82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_SByte.TryParse(value%3a%20%22127%22).html>)
📝 - Benchmark Source		 11.93 ns 14.20 ns 1.19 0.09 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_SByte.TryParse(value: "127") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.197530899075202 > 12.56178043363791. IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -16.861707861582495 (T) = (0 -14.232159728978075) / Math.Sqrt((0.00859080248235329 / (19)) + (0.16584197045771878 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.18292654724110605 = (12.031313154794939 - 14.232159728978075) / 12.031313154794939 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 83bf4b66ed3de5905b2e410a194a0f5f3d924c3d
Compare 82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.BenchE

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Benchstone.BenchI.BenchE.Test.html>)
📝 - Benchmark Source		 245.07 ms 320.11 ms 1.31 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BenchE*'
### Payloads [Baseline]() [Compare]() ### Benchstone.BenchI.BenchE.Test #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 320.1089333333333 > 257.3510205. IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 12:30:38 AM, 5/26/2023 6:22:05 AM, 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 6/13/2023 11:57:38 AM, 6/20/2023 9:17:03 AM, 6/30/2023 8:34:57 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -150.1092793209235 (T) = (0 -322493599.23280424) / Math.Sqrt((9818554358.852018 / (19)) + (2390667044638.784 / (9))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (9) - 2, .025) and -0.3160149151726372 = (245053149.10545596 - 322493599.23280424) / 245053149.10545596 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
EgorBo commented 1 year ago

Regressed from https://github.com/dotnet/runtime/pull/86875 cc @tannergooding

ghost commented 1 year ago

Tagging subscribers to this area: @dotnet/area-system-numerics See info in area-owners.md if you want to be subscribed.

Issue Details
### Run Information Name | Value -- | -- Architecture | x64 OS | Windows 10.0.19042 Queue | OwlWindows Baseline | [83bf4b66ed3de5905b2e410a194a0f5f3d924c3d](https://github.com/dotnet/runtime/commit/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d) Compare | [82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5](https://github.com/dotnet/runtime/commit/82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Diff | [Diff](https://github.com/dotnet/runtime/compare/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d...82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in System.Tests.Perf_Int32 Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- [TryParseSpan - Duration of single invocation]()
📝 - [Benchmark Source]() | 10.99 ns | 14.19 ns | 1.29 | 0.08 | False | | | [ToString - Duration of single invocation]()
📝 - [Benchmark Source]() | 10.90 ns | 14.99 ns | 1.37 | 0.13 | False | | | [TryParse - Duration of single invocation]()
📝 - [Benchmark Source]() | 15.58 ns | 20.40 ns | 1.31 | 0.11 | False | | | [TryParseSpan - Duration of single invocation]()
📝 - [Benchmark Source]() | 16.62 ns | 19.16 ns | 1.15 | 0.09 | False | | | [Parse - Duration of single invocation]()
📝 - [Benchmark Source]() | 14.40 ns | 17.33 ns | 1.20 | 0.06 | False | | | [ToString - Duration of single invocation]()
📝 - [Benchmark Source]() | 8.53 ns | 11.91 ns | 1.40 | 0.12 | False | | | [ParseSpan - Duration of single invocation]()
📝 - [Benchmark Source]() | 11.63 ns | 16.67 ns | 1.43 | 0.09 | False | | | [Parse - Duration of single invocation]()
📝 - [Benchmark Source]() | 11.10 ns | 16.83 ns | 1.52 | 0.12 | False | | | [TryParse - Duration of single invocation]()
📝 - [Benchmark Source]() | 14.38 ns | 19.78 ns | 1.38 | 0.06 | False | | | [TryParseSpan - Duration of single invocation]()
📝 - [Benchmark Source]() | 14.30 ns | 19.68 ns | 1.38 | 0.08 | False | | | ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*' ```
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Int32.TryParseSpan(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.190120288945042 > 11.534074752675743. IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -24.697939964773763 (T) = (0 -14.407504759071363) / Math.Sqrt((0.000210871079473183 / (19)) + (0.1920708851182739 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.31172436769823975 = (10.98363735085067 - 14.407504759071363) / 10.98363735085067 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.ToString(value: 2147483647) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.993700539555588 > 12.368893105520494. IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 6/2/2023 6:53:36 PM, 6/5/2023 10:43:22 AM, 7/12/2023 9:48:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -27.260590010041422 (T) = (0 -15.496795506087762) / Math.Sqrt((0.18115007417294351 / (22)) + (0.08147438563616338 / (7))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (7) - 2, .025) and -0.3297635832937344 = (11.653797487597945 - 15.496795506087762) / 11.653797487597945 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.TryParse(value: "-2147483648") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.396802256987975 > 16.3567919600678. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -26.031347472653543 (T) = (0 -19.70150746050102) / Math.Sqrt((0.011863208373597493 / (19)) + (0.23923420674679818 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.2610591640305737 = (15.622984252008788 - 19.70150746050102) / 15.622984252008788 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.16131198673049 > 17.506325757575464. IsChangePoint: Marked as a change because one of 4/26/2023 6:27:29 AM, 4/26/2023 4:28:25 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -23.009844760978734 (T) = (0 -19.84466188881448) / Math.Sqrt((0.14091681560057817 / (19)) + (0.14438831076899844 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.20687769829617622 = (16.44297671324146 - 19.84466188881448) / 16.44297671324146 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.Parse(value: "2147483647") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.329416854116946 > 15.091691023660475. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -16.312934516703194 (T) = (0 -18.19734938679178) / Math.Sqrt((0.0015169596589323792 / (19)) + (0.548154216803538 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.2658786494521896 = (14.375271590737947 - 18.19734938679178) / 14.375271590737947 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.ToString(value: 12345) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.90771421298525 > 9.010497568013985. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/22/2023 7:55:14 AM, 7/12/2023 9:48:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -21.22220608159406 (T) = (0 -12.28222343184322) / Math.Sqrt((0.07844561528440665 / (22)) + (0.1741739647391373 / (7))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (7) - 2, .025) and -0.4112970353405608 = (8.702791208570343 - 12.28222343184322) / 8.702791208570343 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.ParseSpan(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.66542346740148 > 12.211811377879068. IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -25.408789976545123 (T) = (0 -15.937928289814147) / Math.Sqrt((1.1162861616165135E-05 / (19)) + (0.28747723161934796 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.3704405032137533 = (11.629784914003116 - 15.937928289814147) / 11.629784914003116 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.Parse(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.826361980205 > 11.649755632515522. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -16.09636804454305 (T) = (0 -15.181679741034051) / Math.Sqrt((0.0041244532777064075 / (19)) + (0.5743099533001782 / (9))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (9) - 2, .025) and -0.36665350673470265 = (11.108653119624378 - 15.181679741034051) / 11.108653119624378 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.TryParse(value: "2147483647") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.77535313009241 > 15.093973076160415. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -50.573425619394655 (T) = (0 -19.0281170471599) / Math.Sqrt((0.0034886385822024257 / (19)) + (0.08235326096061854 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.3225208125873535 = (14.387763781148871 - 19.0281170471599) / 14.387763781148871 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int32.TryParseSpan(value: "2147483647") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.680236497246323 > 14.97101717214047. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -71.27114399575764 (T) = (0 -19.312547634887263) / Math.Sqrt((0.0017930930556328644 / (19)) + (0.049113813075204606 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.3533653853715328 = (14.270017427396729 - 19.312547634887263) / 14.270017427396729 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
--- ### Run Information Name | Value -- | -- Architecture | x64 OS | Windows 10.0.19042 Queue | OwlWindows Baseline | [83bf4b66ed3de5905b2e410a194a0f5f3d924c3d](https://github.com/dotnet/runtime/commit/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d) Compare | [82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5](https://github.com/dotnet/runtime/commit/82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Diff | [Diff](https://github.com/dotnet/runtime/compare/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d...82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in System.Memory.Span<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- [EndsWith - Duration of single invocation]()
📝 - [Benchmark Source]() | 19.98 ns | 23.93 ns | 1.20 | 0.11 | False | | | [IndexOfAnyFiveValues - Duration of single invocation]()
📝 - [Benchmark Source]() | 32.27 ns | 43.41 ns | 1.35 | 0.18 | False | | | [StartsWith - Duration of single invocation]()
📝 - [Benchmark Source]() | 17.61 ns | 23.07 ns | 1.31 | 0.17 | False | | | ![graph]() ![graph]() ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Int32>.EndsWith(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.928167560437533 > 20.999560282601973. IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/2/2023 6:53:36 PM, 6/8/2023 8:20:44 PM, 6/22/2023 7:55:14 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -8.54359133045494 (T) = (0 -22.311792180136468) / Math.Sqrt((0.22509452045072115 / (19)) + (0.6775623704422623 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.12112244848097427 = (19.901298212668074 - 22.311792180136468) / 19.901298212668074 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.41256488683741 > 33.98140889328459. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/13/2023 5:43:42 PM, 6/19/2023 6:37:24 PM, 7/5/2023 7:07:19 PM, 7/12/2023 9:48:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -59.5690257957098 (T) = (0 -42.79789455195748) / Math.Sqrt((0.1796580122748577 / (22)) + (0.15216707898017826 / (7))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (7) - 2, .025) and -0.31699264779164527 = (32.49668449077653 - 42.79789455195748) / 32.49668449077653 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.StartsWith(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.071253447888303 > 18.3165331355214. IsChangePoint: Marked as a change because one of 5/16/2023 5:40:18 PM, 6/22/2023 7:55:14 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -7.109646695955824 (T) = (0 -22.868928049019054) / Math.Sqrt((0.1943529829533172 / (19)) + (5.747053214192557 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.3119387828064188 = (17.43139874262978 - 22.868928049019054) / 17.43139874262978 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
--- ### Run Information Name | Value -- | -- Architecture | x64 OS | Windows 10.0.19042 Queue | OwlWindows Baseline | [83bf4b66ed3de5905b2e410a194a0f5f3d924c3d](https://github.com/dotnet/runtime/commit/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d) Compare | [82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5](https://github.com/dotnet/runtime/commit/82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Diff | [Diff](https://github.com/dotnet/runtime/compare/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d...82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in System.Text.Json.Tests.Perf_Get Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- [GetSingle - Duration of single invocation]()
📝 - [Benchmark Source]() | 4.53 Ξs | 4.82 Ξs | 1.06 | 0.02 | False | | | ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*' ```
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Tests.Perf_Get.GetSingle #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.819585599387208 > 4.758078055162659. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -12.816471219622851 (T) = (0 -4857.4633481907085) / Math.Sqrt((1694.7576055906093 / (19)) + (4334.554670054745 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.06419276112964237 = (4564.458174884128 - 4857.4633481907085) / 4564.458174884128 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
--- ### Run Information Name | Value -- | -- Architecture | x64 OS | Windows 10.0.19042 Queue | OwlWindows Baseline | [83bf4b66ed3de5905b2e410a194a0f5f3d924c3d](https://github.com/dotnet/runtime/commit/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d) Compare | [82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5](https://github.com/dotnet/runtime/commit/82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Diff | [Diff](https://github.com/dotnet/runtime/compare/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d...82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in System.Collections.ContainsFalse<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- [List - Duration of single invocation]()
📝 - [Benchmark Source]() | 16.44 Ξs | 20.26 Ξs | 1.23 | 0.10 | False | | | ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsFalse<Int32>.List(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.259371761658027 > 17.259348547581492. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/2/2023 6:53:36 PM, 6/13/2023 11:57:38 AM, 6/19/2023 11:51:49 PM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -202.4740960409093 (T) = (0 -20369.29188027515) / Math.Sqrt((94.10078871264959 / (19)) + (3726.473764407471 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.2393978850339125 = (16434.828658528655 - 20369.29188027515) / 16434.828658528655 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
--- ### Run Information Name | Value -- | -- Architecture | x64 OS | Windows 10.0.19042 Queue | OwlWindows Baseline | [83bf4b66ed3de5905b2e410a194a0f5f3d924c3d](https://github.com/dotnet/runtime/commit/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d) Compare | [82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5](https://github.com/dotnet/runtime/commit/82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Diff | [Diff](https://github.com/dotnet/runtime/compare/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d...82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in System.Collections.ContainsTrue<Int32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- [Queue - Duration of single invocation]()
📝 - [Benchmark Source]() | 13.20 Ξs | 16.68 Ξs | 1.26 | 0.18 | False | | | ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<Int32>*' ```
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<Int32>.Queue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.682858635716826 > 13.861466273593738. IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/2/2023 6:53:36 PM, 6/14/2023 4:56:58 PM, 6/27/2023 10:31:09 PM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -2.232653896944129 (T) = (0 -15998.855719533469) / Math.Sqrt((3352948.64773084 / (19)) + (3292236.2439783234 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.110170783211233 = (14411.166247102861 - 15998.855719533469) / 14411.166247102861 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
--- ### Run Information Name | Value -- | -- Architecture | x64 OS | Windows 10.0.19042 Queue | OwlWindows Baseline | [83bf4b66ed3de5905b2e410a194a0f5f3d924c3d](https://github.com/dotnet/runtime/commit/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d) Compare | [82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5](https://github.com/dotnet/runtime/commit/82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Diff | [Diff](https://github.com/dotnet/runtime/compare/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d...82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in System.Tests.Perf_Int64 Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- [ParseSpan - Duration of single invocation]()
📝 - [Benchmark Source]() | 13.02 ns | 15.32 ns | 1.18 | 0.09 | False | | | [TryParseSpan - Duration of single invocation]()
📝 - [Benchmark Source]() | 12.21 ns | 15.12 ns | 1.24 | 0.06 | False | | | ![graph]() ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*' ```
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Int64.ParseSpan(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.318294192831162 > 13.648609928158619. IsChangePoint: Marked as a change because one of 4/19/2023 8:38:27 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -12.295197589970563 (T) = (0 -15.048104182117617) / Math.Sqrt((0.005490628588379624 / (19)) + (0.269115252900472 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.15574090629500364 = (13.020309396470024 - 15.048104182117617) / 13.020309396470024 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Int64.TryParseSpan(value: "12345") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.117739991647596 > 12.997921566357508. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -30.475514094506334 (T) = (0 -15.025658001356609) / Math.Sqrt((0.0063871462073339935 / (19)) + (0.07486717501870088 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.21860734658284697 = (12.33018826243806 - 15.025658001356609) / 12.33018826243806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
--- ### Run Information Name | Value -- | -- Architecture | x64 OS | Windows 10.0.19042 Queue | OwlWindows Baseline | [83bf4b66ed3de5905b2e410a194a0f5f3d924c3d](https://github.com/dotnet/runtime/commit/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d) Compare | [82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5](https://github.com/dotnet/runtime/commit/82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Diff | [Diff](https://github.com/dotnet/runtime/compare/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d...82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in System.Tests.Perf_SByte Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- [TryParse - Duration of single invocation]()
📝 - [Benchmark Source]() | 11.93 ns | 14.20 ns | 1.19 | 0.09 | False | | | ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*' ```
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_SByte.TryParse(value: "127") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.197530899075202 > 12.56178043363791. IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/13/2023 1:49:08 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -16.861707861582495 (T) = (0 -14.232159728978075) / Math.Sqrt((0.00859080248235329 / (19)) + (0.16584197045771878 / (10))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (10) - 2, .025) and -0.18292654724110605 = (12.031313154794939 - 14.232159728978075) / 12.031313154794939 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
--- ### Run Information Name | Value -- | -- Architecture | x64 OS | Windows 10.0.19042 Queue | OwlWindows Baseline | [83bf4b66ed3de5905b2e410a194a0f5f3d924c3d](https://github.com/dotnet/runtime/commit/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d) Compare | [82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5](https://github.com/dotnet/runtime/commit/82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Diff | [Diff](https://github.com/dotnet/runtime/compare/83bf4b66ed3de5905b2e410a194a0f5f3d924c3d...82bf906968d9bc1ca5b3131c7a7af8f30d5ac2f5) Configs | CompilationMode:tiered, RunKind:micro ### Regressions in Benchstone.BenchI.BenchE Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio -- | -- | -- | -- | -- | -- | -- | -- | -- [Test - Duration of single invocation]()
📝 - [Benchmark Source]() | 245.07 ms | 320.11 ms | 1.31 | 0.05 | False | | | ![graph]() [Test Report]() ### Repro General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BenchE*' ```
### Payloads [Baseline]() [Compare]() ### Benchstone.BenchI.BenchE.Test #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 320.1089333333333 > 257.3510205. IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/2/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 12:30:38 AM, 5/26/2023 6:22:05 AM, 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 6/13/2023 11:57:38 AM, 6/20/2023 9:17:03 AM, 6/30/2023 8:34:57 AM, 7/12/2023 2:12:47 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -150.1092793209235 (T) = (0 -322493599.23280424) / Math.Sqrt((9818554358.852018 / (19)) + (2390667044638.784 / (9))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (9) - 2, .025) and -0.3160149151726372 = (245053149.10545596 - 322493599.23280424) / 245053149.10545596 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Author: performanceautofiler[bot]
Assignees: tannergooding
Labels: `area-System.Numerics`, `os-windows`, `tenet-performance`, `tenet-performance-benchmarks`, `arch-x64`, `untriaged`, `runtime-coreclr`, `needs-area-label`
Milestone: -
tannergooding commented 1 year ago

We expected some small regressions, as in the past. We also notably saw several perf improvements that were linked to from the PR.

Will take a look to see if there is anything small we can do to win back the 4ns lost.