search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x86: 25 Regressions on 4/17/2023 11:29:15 PM #16589

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SingleWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input%3a%20Array).html>) 687.32 ns 742.13 ns 1.08 0.01 False Trace Trace
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.Reverse(input%3a%20IEnumerable).html>) 1.21 μs 1.29 μs 1.07 0.02 False Trace Trace
[ToDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.ToDictionary(input%3a%20Array).html>) 1.16 μs 1.45 μs 1.25 0.00 True 10269.36026936027 13650.809581158837 1.329275555935795) Trace Trace
[SelectToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20Array).html>) 283.11 ns 309.49 ns 1.09 0.02 False Trace Trace
[WhereSingle_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input%3a%20Array).html>) 325.64 ns 386.30 ns 1.19 0.01 False Trace Trace
[ToDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.ToDictionary(input%3a%20IEnumerable).html>) 2.63 μs 3.46 μs 1.32 0.00 True 19329.637096774193 27325.721153846152 1.4136696419616888) Trace Trace
[OrderByThenBy - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.OrderByThenBy(input%3a%20IEnumerable).html>) 4.79 μs 5.13 μs 1.07 0.00 False Trace Trace
[ToDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.ToDictionary(input%3a%20List).html>) 1.26 μs 1.96 μs 1.56 0.03 False 11054.545195810564 14396.868038264973 1.302348290522262) Trace Trace
[Aggregate - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Linq.Tests.Perf_Enumerable.Aggregate(input%3a%20IEnumerable).html>) 631.34 ns 707.83 ns 1.12 0.01 False Trace Trace

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 742.1331540242409 > 719.1477859914767. IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/16/2023 6:27:39 AM, 3/22/2023 7:51:01 PM, 3/25/2023 12:18:49 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -38.167873856198675 (T) = (0 -742.4800602564292) / Math.Sqrt((112.2945691992134 / (35)) + (0.24068786786216953 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.10349642767798926 = (672.8431933565732 - 742.4800602564292) / 672.8431933565732 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_SingleWithPredicate_LastElementMatches_input__Array__baseline_81a16862-2644-46c4-b70c-e624900e8b38.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_SingleWithPredicate_LastElementMatches_input__Array__compare_81a16862-2644-46c4-b70c-e624900e8b38.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_SingleWithPredicate_LastElementMatches_input__Array__baseline_81a16862-2644-46c4-b70c-e624900e8b38.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_SingleWithPredicate_LastElementMatches_input__Array__compare_81a16862-2644-46c4-b70c-e624900e8b38.log) #### System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2921712196467992 > 1.2697033720038682. IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 3/6/2023 11:14:52 AM, 3/22/2023 7:51:01 PM, 4/8/2023 3:09:32 AM, 4/9/2023 11:04:55 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -4.912810878049683 (T) = (0 -1326.8874108797545) / Math.Sqrt((2956.469688176976 / (19)) + (858.6830016543998 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.05573505671026552 = (1256.8375014602775 - 1326.8874108797545) / 1256.8375014602775 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_Reverse_input__IEnumerable__baseline_bc5791bd-962c-4b33-8dff-fdae1a2879f1.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_Reverse_input__IEnumerable__compare_bc5791bd-962c-4b33-8dff-fdae1a2879f1.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_Reverse_input__IEnumerable__baseline_bc5791bd-962c-4b33-8dff-fdae1a2879f1.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_Reverse_input__IEnumerable__compare_bc5791bd-962c-4b33-8dff-fdae1a2879f1.log) #### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4512168758716875 > 1.2239426504672903. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -147.66888441566599 (T) = (0 -1446.22891382335) / Math.Sqrt((6.407984819774586 / (31)) + (20.114497004486413 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.2385892708294462 = (1167.6420488083622 - 1446.22891382335) / 1167.6420488083622 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_ToDictionary_input__Array__baseline_5de87184-5b7c-49a8-aeee-511adab8afa5.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_ToDictionary_input__Array__compare_5de87184-5b7c-49a8-aeee-511adab8afa5.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_ToDictionary_input__Array__baseline_5de87184-5b7c-49a8-aeee-511adab8afa5.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_ToDictionary_input__Array__compare_5de87184-5b7c-49a8-aeee-511adab8afa5.log) #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 309.4871666198295 > 295.420232901797. IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/14/2023 5:47:38 PM, 2/20/2023 10:31:48 AM, 2/23/2023 8:54:44 PM, 4/8/2023 3:09:32 AM, 4/9/2023 11:04:55 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -5.001287388253546 (T) = (0 -310.01794413489836) / Math.Sqrt((210.29524023329319 / (19)) + (1.5033419348737587 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.056939748555194084 = (293.31657226316247 - 310.01794413489836) / 293.31657226316247 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_SelectToArray_input__Array__baseline_1e6d0263-d1f3-414d-a549-9f55394560fa.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_SelectToArray_input__Array__compare_1e6d0263-d1f3-414d-a549-9f55394560fa.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_SelectToArray_input__Array__baseline_1e6d0263-d1f3-414d-a549-9f55394560fa.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_SelectToArray_input__Array__compare_1e6d0263-d1f3-414d-a549-9f55394560fa.log) #### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 386.3009524639247 > 342.0476315586225. IsChangePoint: Marked as a change because one of 1/31/2023 6:07:33 PM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -496.5300622071817 (T) = (0 -386.3250460735877) / Math.Sqrt((0.5007941154746658 / (35)) + (0.001161004053188561 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.18599787262420348 = (325.7383971682714 - 386.3250460735877) / 325.7383971682714 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_WhereSingle_LastElementMatches_input__Array__baseline_d75d0cb6-0c86-4681-ae0a-aca8d5ae85b7.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_WhereSingle_LastElementMatches_input__Array__compare_d75d0cb6-0c86-4681-ae0a-aca8d5ae85b7.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_WhereSingle_LastElementMatches_input__Array__baseline_d75d0cb6-0c86-4681-ae0a-aca8d5ae85b7.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_WhereSingle_LastElementMatches_input__Array__compare_d75d0cb6-0c86-4681-ae0a-aca8d5ae85b7.log) #### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4630773003183064 > 2.7516809903970185. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -110.00122497772064 (T) = (0 -3480.650101112146) / Math.Sqrt((274.76194265404814 / (31)) + (318.9453782598576 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.33136146246948917 = (2614.3539521235857 - 3480.650101112146) / 2614.3539521235857 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_ToDictionary_input__IEnumerable__baseline_a88a2429-731e-4d72-8ad1-8b801eb94a63.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_ToDictionary_input__IEnumerable__compare_a88a2429-731e-4d72-8ad1-8b801eb94a63.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_ToDictionary_input__IEnumerable__baseline_a88a2429-731e-4d72-8ad1-8b801eb94a63.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_ToDictionary_input__IEnumerable__compare_a88a2429-731e-4d72-8ad1-8b801eb94a63.log) #### System.Linq.Tests.Perf_Enumerable.OrderByThenBy(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.129179247347116 > 5.056428696335412. IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/20/2023 8:11:49 PM, 3/6/2023 11:14:52 AM, 4/8/2023 3:09:32 AM, 4/9/2023 11:04:55 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -14.176942287949263 (T) = (0 -5170.481460779483) / Math.Sqrt((6678.321721369761 / (19)) + (1231.121111534524 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.05952924188864758 = (4879.979953703704 - 5170.481460779483) / 4879.979953703704 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_OrderByThenBy_input__IEnumerable__baseline_051e45a7-247d-4a5c-a68f-99c190b1cc26.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_OrderByThenBy_input__IEnumerable__compare_051e45a7-247d-4a5c-a68f-99c190b1cc26.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_OrderByThenBy_input__IEnumerable__baseline_051e45a7-247d-4a5c-a68f-99c190b1cc26.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_OrderByThenBy_input__IEnumerable__compare_051e45a7-247d-4a5c-a68f-99c190b1cc26.log) #### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9585088902803804 > 1.321621428021972. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/17/2023 9:25:32 PM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -72.19100000837524 (T) = (0 -1963.1618421741753) / Math.Sqrt((2860.5127334342533 / (31)) + (12.108514507447271 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.5554043478801396 = (1262.1553005491905 - 1963.1618421741753) / 1262.1553005491905 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_ToDictionary_input__List__baseline_ff3ec5c9-3622-4207-953e-aaa97002bdca.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_ToDictionary_input__List__compare_ff3ec5c9-3622-4207-953e-aaa97002bdca.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_ToDictionary_input__List__baseline_ff3ec5c9-3622-4207-953e-aaa97002bdca.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_ToDictionary_input__List__compare_ff3ec5c9-3622-4207-953e-aaa97002bdca.log) #### System.Linq.Tests.Perf_Enumerable.Aggregate(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 707.8256182820536 > 663.2953353526278. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/16/2023 6:27:39 AM, 3/17/2023 10:06:19 AM, 4/9/2023 11:04:55 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -12.52504005559656 (T) = (0 -690.592427856371) / Math.Sqrt((51.22884319718801 / (19)) + (264.0742506406732 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.08176210415619094 = (638.3958406410023 - 690.592427856371) / 638.3958406410023 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_Aggregate_input__IEnumerable__baseline_c0d6835e-365d-44d8-af6b-73d07e4305e8.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Linq_Tests_Perf_Enumerable_Aggregate_input__IEnumerable__compare_c0d6835e-365d-44d8-af6b-73d07e4305e8.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_Aggregate_input__IEnumerable__baseline_c0d6835e-365d-44d8-af6b-73d07e4305e8.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Linq_Tests_Perf_Enumerable_Aggregate_input__IEnumerable__compare_c0d6835e-365d-44d8-af6b-73d07e4305e8.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.IndexerSet(Int32).IList(Size%3a%20512).html>) 2.04 μs 2.48 μs 1.22 0.04 False Trace Trace
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.IndexerSet(Int32).Dictionary(Size%3a%20512).html>) 5.55 μs 11.10 μs 2.00 0.11 True Trace Trace

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<Int32>.IList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4786203275376186 > 2.023815974301931. IsChangePoint: Marked as a change because one of 2/16/2023 6:27:39 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 4/9/2023 7:17:29 PM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -17.764417153847344 (T) = (0 -2500.7641283384696) / Math.Sqrt((11933.154204033686 / (35)) + (980.6958278155247 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.2575645948417182 = (1988.577078741013 - 2500.7641283384696) / 1988.577078741013 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_IndexerSet_Int32__IList_Size__512__baseline_3f10e381-2c69-4096-bfae-4e6d830a4ef6.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_IndexerSet_Int32__IList_Size__512__compare_3f10e381-2c69-4096-bfae-4e6d830a4ef6.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_IndexerSet_Int32__IList_Size__512__baseline_3f10e381-2c69-4096-bfae-4e6d830a4ef6.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_IndexerSet_Int32__IList_Size__512__compare_3f10e381-2c69-4096-bfae-4e6d830a4ef6.log) #### System.Collections.IndexerSet<Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.101486278684646 > 6.0961168814265. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -61.09340381964261 (T) = (0 -11030.930274467175) / Math.Sqrt((14675.221448732189 / (31)) + (43652.67559164946 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.9513302728146595 = (5653.03087239856 - 11030.930274467175) / 5653.03087239856 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_IndexerSet_Int32__Dictionary_Size__512__baseline_b6132a20-afc9-4f01-81b1-fb6ac30da282.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_IndexerSet_Int32__Dictionary_Size__512__compare_b6132a20-afc9-4f01-81b1-fb6ac30da282.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_IndexerSet_Int32__Dictionary_Size__512__baseline_b6132a20-afc9-4f01-81b1-fb6ac30da282.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_IndexerSet_Int32__Dictionary_Size__512__compare_b6132a20-afc9-4f01-81b1-fb6ac30da282.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_Version.Parse2.html>) 30.05 ns 35.63 ns 1.19 0.02 False Trace Trace
[ToString2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_Version.ToString2.html>) 26.61 ns 32.85 ns 1.23 0.10 True 303.5153621136681 332.1626232942149 1.0943848804918752) Trace Trace
[TryParse2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_Version.TryParse2.html>) 31.10 ns 34.02 ns 1.09 0.01 False 350.21489558093924 371.8697016236221 1.0618329097817547) Trace Trace
[TryFormatL - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_Version.TryFormatL.html>) 50.34 ns 57.04 ns 1.13 0.11 False 575.8733414770937 673.7820131961599 1.1700177185975202) Trace Trace
[TryFormat4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_Version.TryFormat4.html>) 27.90 ns 35.81 ns 1.28 0.15 False 333.6935928238383 389.0414208158373 1.1658642214962094) Trace Trace
[ToString3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_Version.ToString3.html>) 34.55 ns 42.41 ns 1.23 0.10 True 385.97603122220954 429.5356778054853 1.1128558331597491) Trace Trace
[Parse3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_Version.Parse3.html>) 34.98 ns 37.80 ns 1.08 0.01 False 440.8608564691877 471.56350353902974 1.0696424883709037) Trace Trace
[TryParse3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_Version.TryParse3.html>) 34.92 ns 37.93 ns 1.09 0.01 False 436.1012597185802 459.30929756212265 1.053217085083677) Trace Trace
[ToString4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_Version.ToString4.html>) 40.31 ns 48.51 ns 1.20 0.10 True 456.61151298467337 513.1194857589613 1.1237550328175467) Trace Trace

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Version.Parse2 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.63497330890304 > 31.582559903363. IsChangePoint: Marked as a change because one of 2/20/2023 10:31:48 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -42.120209924645174 (T) = (0 -35.52388439753236) / Math.Sqrt((0.12555998850903047 / (35)) + (0.02468149245904515 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.1759750935475846 = (30.208024466204332 - 35.52388439753236) / 30.208024466204332 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_Parse2_baseline_69431c56-a616-4a63-95ec-50870cf9610a.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_Parse2_compare_69431c56-a616-4a63-95ec-50870cf9610a.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_Parse2_baseline_69431c56-a616-4a63-95ec-50870cf9610a.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_Parse2_compare_69431c56-a616-4a63-95ec-50870cf9610a.log) #### System.Tests.Perf_Version.ToString2 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.848805795997556 > 27.96145460473401. IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -71.4244183888537 (T) = (0 -33.110038597305916) / Math.Sqrt((0.07297783281162958 / (19)) + (0.07419189705605007 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.23838109185555825 = (26.73655049730668 - 33.110038597305916) / 26.73655049730668 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_ToString2_baseline_efdf6cea-6690-478d-9ff9-8ca0013b0165.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_ToString2_compare_efdf6cea-6690-478d-9ff9-8ca0013b0165.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_ToString2_baseline_efdf6cea-6690-478d-9ff9-8ca0013b0165.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_ToString2_compare_efdf6cea-6690-478d-9ff9-8ca0013b0165.log) #### System.Tests.Perf_Version.TryParse2 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 34.022396331333255 > 32.555452863459195. IsChangePoint: Marked as a change because one of 2/8/2023 3:55:57 PM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -6.723832335095343 (T) = (0 -33.606032708484705) / Math.Sqrt((0.1935692587393255 / (35)) + (0.34671733286313344 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.0924468290271121 = (30.762167837873488 - 33.606032708484705) / 30.762167837873488 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_TryParse2_baseline_6065d5ad-de12-40b0-a893-bdf6362788c9.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_TryParse2_compare_6065d5ad-de12-40b0-a893-bdf6362788c9.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_TryParse2_baseline_6065d5ad-de12-40b0-a893-bdf6362788c9.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_TryParse2_compare_6065d5ad-de12-40b0-a893-bdf6362788c9.log) #### System.Tests.Perf_Version.TryFormatL ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.043168608739464 > 51.03706927264439. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 4/9/2023 11:04:55 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -9.47162342915593 (T) = (0 -56.06360220953814) / Math.Sqrt((0.983494560689723 / (19)) + (10.47057167468287 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.1553523882176211 = (48.52511041763481 - 56.06360220953814) / 48.52511041763481 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_TryFormatL_baseline_0773ffe5-9ca4-46be-a313-01edb581a602.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_TryFormatL_compare_0773ffe5-9ca4-46be-a313-01edb581a602.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_TryFormatL_baseline_0773ffe5-9ca4-46be-a313-01edb581a602.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_TryFormatL_compare_0773ffe5-9ca4-46be-a313-01edb581a602.log) #### System.Tests.Perf_Version.TryFormat4 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.808085732991245 > 29.158975904992126. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 4/9/2023 11:04:55 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -30.98588372119761 (T) = (0 -36.27116511457891) / Math.Sqrt((1.0508076824103667 / (19)) + (0.24370380488724167 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.2889081727880192 = (28.141000173907862 - 36.27116511457891) / 28.141000173907862 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_TryFormat4_baseline_a58bc786-bebc-4911-b48b-20ff65d85448.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_TryFormat4_compare_a58bc786-bebc-4911-b48b-20ff65d85448.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_TryFormat4_baseline_a58bc786-bebc-4911-b48b-20ff65d85448.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_TryFormat4_compare_a58bc786-bebc-4911-b48b-20ff65d85448.log) #### System.Tests.Perf_Version.ToString3 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 42.41333146302309 > 36.282351531116916. IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -51.35578012014476 (T) = (0 -42.79686669803291) / Math.Sqrt((0.24057900807593433 / (19)) + (0.22002806466838287 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.23350379969134305 = (34.69536673396699 - 42.79686669803291) / 34.69536673396699 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_ToString3_baseline_805a1e3f-d144-43f8-b9b6-086ef6cd0376.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_ToString3_compare_805a1e3f-d144-43f8-b9b6-086ef6cd0376.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_ToString3_baseline_805a1e3f-d144-43f8-b9b6-086ef6cd0376.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_ToString3_compare_805a1e3f-d144-43f8-b9b6-086ef6cd0376.log) #### System.Tests.Perf_Version.Parse3 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.799994847063545 > 36.76651454290786. IsChangePoint: Marked as a change because one of 1/30/2023 9:42:46 PM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -10.436787261376738 (T) = (0 -38.08201256493621) / Math.Sqrt((0.2119628799519765 / (35)) + (0.15906798638820518 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.08716761224670166 = (35.0286488816911 - 38.08201256493621) / 35.0286488816911 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_Parse3_baseline_06c5a01f-b2e5-434f-827f-0008063cf7d4.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_Parse3_compare_06c5a01f-b2e5-434f-827f-0008063cf7d4.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_Parse3_baseline_06c5a01f-b2e5-434f-827f-0008063cf7d4.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_Parse3_compare_06c5a01f-b2e5-434f-827f-0008063cf7d4.log) #### System.Tests.Perf_Version.TryParse3 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.92522506553221 > 36.85191827417962. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -12.960366354269862 (T) = (0 -38.14270685671398) / Math.Sqrt((0.17210006235459546 / (35)) + (0.09459665899126059 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.0841795483745918 = (35.1811717108091 - 38.14270685671398) / 35.1811717108091 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_TryParse3_baseline_ea137e85-640c-49f8-92bf-7de0abacb5c2.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_TryParse3_compare_ea137e85-640c-49f8-92bf-7de0abacb5c2.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_TryParse3_baseline_ea137e85-640c-49f8-92bf-7de0abacb5c2.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_TryParse3_compare_ea137e85-640c-49f8-92bf-7de0abacb5c2.log) #### System.Tests.Perf_Version.ToString4 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 48.51299570315743 > 42.61798723409752. IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -20.306360332739892 (T) = (0 -49.703360496219624) / Math.Sqrt((2.719567293850132 / (19)) + (0.7050306544921994 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.2112976707317483 = (41.03315122053664 - 49.703360496219624) / 41.03315122053664 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_ToString4_baseline_e50bc08b-8bff-4564-8173-044e37ce5c33.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_Version_ToString4_compare_e50bc08b-8bff-4564-8173-044e37ce5c33.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_ToString4_baseline_e50bc08b-8bff-4564-8173-044e37ce5c33.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_Version_ToString4_compare_e50bc08b-8bff-4564-8173-044e37ce5c33.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.IterateFor(Int32).IList(Size%3a%20512).html>) 1.93 μs 2.31 μs 1.19 0.01 False Trace Trace

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<Int32>.IList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3066039703572834 > 2.0247756288580248. IsChangePoint: Marked as a change because one of 1/30/2023 9:42:46 PM, 2/9/2023 4:27:10 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -40.829776648814544 (T) = (0 -2306.21243844898) / Math.Sqrt((2713.9860937127924 / (35)) + (0.30659447043916915 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.18491066166789652 = (1946.317568957244 - 2306.21243844898) / 1946.317568957244 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_IterateFor_Int32__IList_Size__512__baseline_a64495a6-1abb-42c8-8401-a2ee4f36fbaa.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_IterateFor_Int32__IList_Size__512__compare_a64495a6-1abb-42c8-8401-a2ee4f36fbaa.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_IterateFor_Int32__IList_Size__512__baseline_a64495a6-1abb-42c8-8401-a2ee4f36fbaa.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_IterateFor_Int32__IList_Size__512__compare_a64495a6-1abb-42c8-8401-a2ee4f36fbaa.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_SByte.Parse(value%3a%20%22127%22).html>) 11.55 ns 15.84 ns 1.37 0.53 False Trace Trace

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_SByte.Parse(value: "127") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.837222682631763 > 12.429106467938425. IsChangePoint: Marked as a change because one of 2/16/2023 4:50:02 PM, 3/9/2023 1:54:27 PM, 3/15/2023 5:33:41 AM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -13.201121123876282 (T) = (0 -16.07097063840175) / Math.Sqrt((1.1025354220875787 / (35)) + (0.10927621365329464 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.31766939855654497 = (12.196512005216837 - 16.07097063840175) / 12.196512005216837 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_SByte_Parse_value___127___baseline_f9875c76-d21c-4799-9e63-ee943bd5d617.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_SByte_Parse_value___127___compare_f9875c76-d21c-4799-9e63-ee943bd5d617.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_SByte_Parse_value___127___baseline_f9875c76-d21c-4799-9e63-ee943bd5d617.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_SByte_Parse_value___127___compare_f9875c76-d21c-4799-9e63-ee943bd5d617.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CreateAddAndRemove<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.CreateAddAndRemove(String).LinkedList(Size%3a%20512).html>) 19.30 μs 21.81 μs 1.13 0.09 False Trace Trace

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndRemove<String>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.808069336521225 > 20.26198933357441. IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -51.49141984216445 (T) = (0 -21813.108064355452) / Math.Sqrt((69260.48483862348 / (35)) + (50.77755637484308 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.11816845506818947 = (19507.890752494193 - 21813.108064355452) / 19507.890752494193 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_CreateAddAndRemove_String__LinkedList_Size__512__baseline_be960dae-3ecd-44e7-a420-d3b52da15102.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_CreateAddAndRemove_String__LinkedList_Size__512__compare_be960dae-3ecd-44e7-a420-d3b52da15102.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_CreateAddAndRemove_String__LinkedList_Size__512__baseline_be960dae-3ecd-44e7-a420-d3b52da15102.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_CreateAddAndRemove_String__LinkedList_Size__512__compare_be960dae-3ecd-44e7-a420-d3b52da15102.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorDefaultSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.CtorDefaultSize(String).SortedList.html>) 32.03 ns 35.72 ns 1.12 0.10 False Trace Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorDefaultSize<String>.SortedList ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.72480248365204 > 32.503698406998204. IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 3/1/2023 3:09:21 PM, 3/15/2023 5:33:41 AM, 3/16/2023 9:46:36 PM, 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -4.35897110644052 (T) = (0 -37.20342824873856) / Math.Sqrt((0.20287045602946802 / (35)) + (4.3726683063553695 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.20988319746475886 = (30.749603206901476 - 37.20342824873856) / 30.749603206901476 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_CtorDefaultSize_String__SortedList_baseline_0a106d9c-29fc-4601-952b-68752d39c995.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_CtorDefaultSize_String__SortedList_compare_0a106d9c-29fc-4601-952b-68752d39c995.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_CtorDefaultSize_String__SortedList_baseline_0a106d9c-29fc-4601-952b-68752d39c995.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_CtorDefaultSize_String__SortedList_compare_0a106d9c-29fc-4601-952b-68752d39c995.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_UInt32.TryParseHex(value%3a%20%22FFFFFFFF%22).html>) 13.05 ns 14.36 ns 1.10 0.00 False Trace Trace

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt32.TryParseHex(value: "FFFFFFFF") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.360286739346826 > 13.67794857015954. IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -165.78613693043258 (T) = (0 -14.353239157715702) / Math.Sqrt((0.00046546936481181784 / (35)) + (9.933681369468029E-05 / (2))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (2) - 2, .025) and -0.10090340962541802 = (13.037691619648436 - 14.353239157715702) / 13.037691619648436 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_UInt32_TryParseHex_value___FFFFFFFF___baseline_25cfc4ae-3d16-40e0-8ca7-c0bf362756d8.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_UInt32_TryParseHex_value___FFFFFFFF___compare_25cfc4ae-3d16-40e0-8ca7-c0bf362756d8.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_UInt32_TryParseHex_value___FFFFFFFF___baseline_25cfc4ae-3d16-40e0-8ca7-c0bf362756d8.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_UInt32_TryParseHex_value___FFFFFFFF___compare_25cfc4ae-3d16-40e0-8ca7-c0bf362756d8.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)