search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.47k stars 4.76k forks source link

Regressions from Roslyn update #80744

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(Byte).GetHashCodeBenchmark.html>) 79.04 ns 105.44 ns 1.33 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 105.4351294570456 > 83.15154770429102. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -38.87873117596192 (T) = (0 -105.35273354069511) / Math.Sqrt((3.992587969600447 / (23)) + (7.861876182668589 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.31787460021963265 = (79.94139466921767 - 105.35273354069511) / 79.94139466921767 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(UInt16).GetHashCodeBenchmark.html>) 39.22 ns 51.70 ns 1.32 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 51.69693024643556 > 41.1064538719314. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -36.663764455094125 (T) = (0 -51.68067070574212) / Math.Sqrt((1.65991615061608 / (22)) + (1.9274338623056948 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.35719061117778034 = (38.07915430603609 - 51.68067070574212) / 38.07915430603609 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).GetHashCodeBenchmark.html>) 9.48 ns 12.98 ns 1.37 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.97857567164073 > 9.912366475799981. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -93.87222615864435 (T) = (0 -13.031390688994902) / Math.Sqrt((0.011639596703320161 / (23)) + (0.027995569909519624 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.3706358320006545 = (9.50755144783686 - 13.031390688994902) / 9.50755144783686 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(Double).GetHashCodeBenchmark.html>) 13.38 ns 14.95 ns 1.12 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.948886232048702 > 14.071781964405073. IsChangePoint: Marked as a change because one of 11/2/2022 4:25:40 PM, 12/14/2022 5:20:21 PM, 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -23.26476081715157 (T) = (0 -14.987631064522386) / Math.Sqrt((0.05384720602955832 / (23)) + (0.045076784078112886 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10574021657368117 = (13.554387223939488 - 14.987631064522386) / 13.554387223939488 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Tests.Perf_HashCode

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Combine_7 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_HashCode.Combine_7.html>) 96.03 μs 105.47 μs 1.10 0.00 False
[Combine_2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_HashCode.Combine_2.html>) 56.55 μs 63.40 μs 1.12 0.00 False
[Combine_1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_HashCode.Combine_1.html>) 46.17 μs 52.16 μs 1.13 0.00 False
[Combine_5 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_HashCode.Combine_5.html>) 74.27 μs 82.59 μs 1.11 0.00 False
[Combine_4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_HashCode.Combine_4.html>) 63.99 μs 71.89 μs 1.12 0.00 False
[Combine_6 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_HashCode.Combine_6.html>) 82.93 μs 93.13 μs 1.12 0.00 False
[Combine_8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_HashCode.Combine_8.html>) 82.16 μs 90.71 μs 1.10 0.00 False
[Combine_3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_HashCode.Combine_3.html>) 69.56 μs 74.71 μs 1.07 0.00 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_HashCode*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_HashCode.Combine_7 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 105.4663142677373 > 100.81468016679449. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -161.18619752295376 (T) = (0 -105545.67733509371) / Math.Sqrt((6585.849273667598 / (23)) + (98379.09306130768 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.09869467137568116 = (96064.61202086239 - 105545.67733509371) / 96064.61202086239 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 63.40146640519568 > 59.350448427346585. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -521.8008790934781 (T) = (0 -63448.79334557399) / Math.Sqrt((1550.9166089246503 / (23)) + (3334.649210823573 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12207380790148624 = (56546.006954958306 - 63448.79334557399) / 56546.006954958306 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_1 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 52.161184375000005 > 48.49616808904867. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -474.7956288486027 (T) = (0 -52200.563625643365) / Math.Sqrt((2370.9048049895378 / (22)) + (1632.5099591971323 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.13020631808142258 = (46186.756161703495 - 52200.563625643365) / 46186.756161703495 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_5 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.59137885338346 > 77.98060776806872. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -73.22615233862166 (T) = (0 -82505.62178616505) / Math.Sqrt((104660.75114867605 / (23)) + (244552.67765220546 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10986241775819691 = (74338.6031151659 - 82505.62178616505) / 74338.6031151659 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 71.89388650884665 > 67.18613039001559. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -281.6459962191387 (T) = (0 -71670.10661699313) / Math.Sqrt((709.6575365608301 / (23)) + (22155.12236267368 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12019719537860528 = (63979.90185359284 - 71670.10661699313) / 63979.90185359284 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_6 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 93.12999140625 > 87.02870333416007. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -115.48185935902607 (T) = (0 -93069.88297768412) / Math.Sqrt((3333.509569601689 / (23)) + (235566.862937844 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12257404511481351 = (82907.56710677843 - 93069.88297768412) / 82907.56710677843 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_8 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 90.71372703516377 > 86.27665725675548. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -535.3374364970878 (T) = (0 -90757.96576528928) / Math.Sqrt((3413.288835681361 / (23)) + (3346.5812596985775 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10429181728794486 = (82186.57817114303 - 90757.96576528928) / 82186.57817114303 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 74.7060919059011 > 73.07507285416666. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -140.11060272122677 (T) = (0 -74609.6146257151) / Math.Sqrt((1174.072760930738 / (23)) + (38213.510624717346 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.07213825342420208 = (69589.54629911434 - 74609.6146257151) / 69589.54629911434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(Int64).GetHashCodeBenchmark.html>) 10.81 ns 13.50 ns 1.25 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.501432980336178 > 11.349420695138868. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -53.84020803276331 (T) = (0 -13.53650726544601) / Math.Sqrt((0.008029975980635195 / (24)) + (0.06952606554272087 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.2530132020379094 = (10.803164119444345 - 13.53650726544601) / 10.803164119444345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).GetHashCodeBenchmark.html>) 38.33 ns 54.55 ns 1.42 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 54.54619767782453 > 40.21929684002488. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -82.21235737703599 (T) = (0 -54.1115179051457) / Math.Sqrt((0.06252101293171532 / (24)) + (1.0797309196087996 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.4163482868152744 = (38.20495171199592 - 54.1115179051457) / 38.20495171199592 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).GetHashCodeBenchmark.html>) 19.54 ns 24.49 ns 1.25 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.489060082989262 > 20.74241711124787. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -24.00227051035366 (T) = (0 -24.845609585691953) / Math.Sqrt((0.07136336798950783 / (23)) + (1.2121066048338698 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.2475992129108641 = (19.91473650237632 - 24.845609585691953) / 19.91473650237632 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(UInt32).GetHashCodeBenchmark.html>) 20.29 ns 23.33 ns 1.15 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.331260832845107 > 21.056982512327213. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -25.472944182933528 (T) = (0 -23.645774694088463) / Math.Sqrt((0.1302430376724827 / (23)) + (0.4615801379454354 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.1826468503281096 = (19.993943828225863 - 23.645774694088463) / 19.993943828225863 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(UInt64).GetHashCodeBenchmark.html>) 10.81 ns 13.54 ns 1.25 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.536802932034956 > 11.35145753448431. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -8.574082760730478 (T) = (0 -13.506688811973898) / Math.Sqrt((1.7762903502862928 / (23)) + (0.10097248716469229 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.21965381121373115 = (11.074198832316851 - 13.506688811973898) / 11.074198832316851 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).GetHashCodeBenchmark.html>) 6.14 ns 8.24 ns 1.34 0.20 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.243436857714752 > 6.447982991414564. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -8.389320644851091 (T) = (0 -8.063495790669068) / Math.Sqrt((0.03524392530140337 / (24)) + (1.7697574079138925 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.33647761884757654 = (6.033393808436607 - 8.063495790669068) / 6.033393808436607 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(Int16).GetHashCodeBenchmark.html>) 37.85 ns 51.66 ns 1.37 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 51.66120195005749 > 39.72074575315014. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -39.559854435578245 (T) = (0 -51.881876605118066) / Math.Sqrt((0.8104291630112155 / (23)) + (2.8603275361064218 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.37413871248681874 = (37.75592386246504 - 51.881876605118066) / 37.75592386246504 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(Int32).GetHashCodeBenchmark.html>) 20.03 ns 23.32 ns 1.16 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.320096686065867 > 21.10859220137375. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -27.41942871555706 (T) = (0 -23.556103451989134) / Math.Sqrt((0.045964928710745 / (23)) + (0.4321694720681641 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.17227431687018183 = (20.094361117524805 - 23.556103451989134) / 20.094361117524805 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).GetHashCodeBenchmark.html>) 38.11 ns 54.62 ns 1.43 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 54.619012798368786 > 40.02805183024191. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -94.84377908284456 (T) = (0 -54.19152874424385) / Math.Sqrt((0.03709778719896129 / (24)) + (0.848027256877123 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.42355745765139086 = (38.06767928682692 - 54.19152874424385) / 38.06767928682692 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorOf(SByte).GetHashCodeBenchmark.html>) 79.26 ns 105.48 ns 1.33 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<SByte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 105.48442802319262 > 83.21140817384584. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -38.47455636359471 (T) = (0 -105.38424496933936) / Math.Sqrt((4.656914615225362 / (23)) + (7.935677422340858 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.32837623290392604 = (79.333130448263 - 105.38424496933936) / 79.333130448263 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).GetHashCodeBenchmark.html>) 5.77 ns 7.44 ns 1.29 0.16 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.437152732708328 > 6.263572225291969. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -18.5845985828341 (T) = (0 -7.6594885997405475) / Math.Sqrt((0.10212762220395216 / (24)) + (0.10276806239422005 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.26760735431885063 = (6.042477249476339 - 7.6594885997405475) / 6.042477249476339 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).GetHashCodeBenchmark.html>) 19.68 ns 24.73 ns 1.26 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.72658991657417 > 20.91314539556719. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -19.432827146762506 (T) = (0 -25.219160386806106) / Math.Sqrt((0.04376875150126903 / (23)) + (2.1488050817985904 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.2588724915109568 = (20.033133265575536 - 25.219160386806106) / 20.033133265575536 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).GetHashCodeBenchmark.html>) 9.41 ns 13.56 ns 1.44 0.12 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.561207618701866 > 9.87549907306126. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -11.216464759854135 (T) = (0 -13.204830589160034) / Math.Sqrt((2.0464586540799097 / (23)) + (0.11329716342766691 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.3486623636520238 = (9.7910573802941 - 13.204830589160034) / 9.7910573802941 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
dotnet-issue-labeler[bot] commented 1 year ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

EgorBo commented 1 year ago

Windows-x64 https://github.com/dotnet/perf-autofiling-issues/issues/11738

EgorBo commented 1 year ago

IL diff for HashCode.Combine:

image

ghost commented 1 year ago

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak See info in area-owners.md if you want to be subscribed.

Issue Details
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Numerics.Tests.Perf_VectorOf<Byte> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 79.04 ns | 105.44 ns | 1.33 | 0.06 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Byte>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 105.4351294570456 > 83.15154770429102. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -38.87873117596192 (T) = (0 -105.35273354069511) / Math.Sqrt((3.992587969600447 / (23)) + (7.861876182668589 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.31787460021963265 = (79.94139466921767 - 105.35273354069511) / 79.94139466921767 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 39.22 ns | 51.70 ns | 1.32 | 0.05 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt16>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 51.69693024643556 > 41.1064538719314. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -36.663764455094125 (T) = (0 -51.68067070574212) / Math.Sqrt((1.65991615061608 / (22)) + (1.9274338623056948 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.35719061117778034 = (38.07915430603609 - 51.68067070574212) / 38.07915430603609 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 9.48 ns | 12.98 ns | 1.37 | 0.07 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.97857567164073 > 9.912366475799981. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -93.87222615864435 (T) = (0 -13.031390688994902) / Math.Sqrt((0.011639596703320161 / (23)) + (0.027995569909519624 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.3706358320006545 = (9.50755144783686 - 13.031390688994902) / 9.50755144783686 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Numerics.Tests.Perf_VectorOf<Double> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 13.38 ns | 14.95 ns | 1.12 | 0.03 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Double>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.948886232048702 > 14.071781964405073. IsChangePoint: Marked as a change because one of 11/2/2022 4:25:40 PM, 12/14/2022 5:20:21 PM, 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -23.26476081715157 (T) = (0 -14.987631064522386) / Math.Sqrt((0.05384720602955832 / (23)) + (0.045076784078112886 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10574021657368117 = (13.554387223939488 - 14.987631064522386) / 13.554387223939488 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Tests.Perf_HashCode Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [Combine_7 - Duration of single invocation]() | 96.03 μs | 105.47 μs | 1.10 | 0.00 | False | | | [Combine_2 - Duration of single invocation]() | 56.55 μs | 63.40 μs | 1.12 | 0.00 | False | | | [Combine_1 - Duration of single invocation]() | 46.17 μs | 52.16 μs | 1.13 | 0.00 | False | | | [Combine_5 - Duration of single invocation]() | 74.27 μs | 82.59 μs | 1.11 | 0.00 | False | | | [Combine_4 - Duration of single invocation]() | 63.99 μs | 71.89 μs | 1.12 | 0.00 | False | | | [Combine_6 - Duration of single invocation]() | 82.93 μs | 93.13 μs | 1.12 | 0.00 | False | | | [Combine_8 - Duration of single invocation]() | 82.16 μs | 90.71 μs | 1.10 | 0.00 | False | | | [Combine_3 - Duration of single invocation]() | 69.56 μs | 74.71 μs | 1.07 | 0.00 | False | | | ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_HashCode*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_HashCode.Combine_7 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 105.4663142677373 > 100.81468016679449. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -161.18619752295376 (T) = (0 -105545.67733509371) / Math.Sqrt((6585.849273667598 / (23)) + (98379.09306130768 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.09869467137568116 = (96064.61202086239 - 105545.67733509371) / 96064.61202086239 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 63.40146640519568 > 59.350448427346585. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -521.8008790934781 (T) = (0 -63448.79334557399) / Math.Sqrt((1550.9166089246503 / (23)) + (3334.649210823573 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12207380790148624 = (56546.006954958306 - 63448.79334557399) / 56546.006954958306 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_1 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 52.161184375000005 > 48.49616808904867. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -474.7956288486027 (T) = (0 -52200.563625643365) / Math.Sqrt((2370.9048049895378 / (22)) + (1632.5099591971323 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.13020631808142258 = (46186.756161703495 - 52200.563625643365) / 46186.756161703495 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_5 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.59137885338346 > 77.98060776806872. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -73.22615233862166 (T) = (0 -82505.62178616505) / Math.Sqrt((104660.75114867605 / (23)) + (244552.67765220546 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10986241775819691 = (74338.6031151659 - 82505.62178616505) / 74338.6031151659 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 71.89388650884665 > 67.18613039001559. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -281.6459962191387 (T) = (0 -71670.10661699313) / Math.Sqrt((709.6575365608301 / (23)) + (22155.12236267368 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12019719537860528 = (63979.90185359284 - 71670.10661699313) / 63979.90185359284 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_6 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 93.12999140625 > 87.02870333416007. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -115.48185935902607 (T) = (0 -93069.88297768412) / Math.Sqrt((3333.509569601689 / (23)) + (235566.862937844 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12257404511481351 = (82907.56710677843 - 93069.88297768412) / 82907.56710677843 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_8 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 90.71372703516377 > 86.27665725675548. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -535.3374364970878 (T) = (0 -90757.96576528928) / Math.Sqrt((3413.288835681361 / (23)) + (3346.5812596985775 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10429181728794486 = (82186.57817114303 - 90757.96576528928) / 82186.57817114303 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_HashCode.Combine_3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 74.7060919059011 > 73.07507285416666. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -140.11060272122677 (T) = (0 -74609.6146257151) / Math.Sqrt((1174.072760930738 / (23)) + (38213.510624717346 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.07213825342420208 = (69589.54629911434 - 74609.6146257151) / 69589.54629911434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Numerics.Tests.Perf_VectorOf<Int64> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 10.81 ns | 13.50 ns | 1.25 | 0.03 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int64>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.501432980336178 > 11.349420695138868. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -53.84020803276331 (T) = (0 -13.53650726544601) / Math.Sqrt((0.008029975980635195 / (24)) + (0.06952606554272087 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.2530132020379094 = (10.803164119444345 - 13.53650726544601) / 10.803164119444345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 38.33 ns | 54.55 ns | 1.42 | 0.02 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 54.54619767782453 > 40.21929684002488. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -82.21235737703599 (T) = (0 -54.1115179051457) / Math.Sqrt((0.06252101293171532 / (24)) + (1.0797309196087996 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.4163482868152744 = (38.20495171199592 - 54.1115179051457) / 38.20495171199592 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 19.54 ns | 24.49 ns | 1.25 | 0.03 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.489060082989262 > 20.74241711124787. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -24.00227051035366 (T) = (0 -24.845609585691953) / Math.Sqrt((0.07136336798950783 / (23)) + (1.2121066048338698 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.2475992129108641 = (19.91473650237632 - 24.845609585691953) / 19.91473650237632 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 20.29 ns | 23.33 ns | 1.15 | 0.04 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt32>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.331260832845107 > 21.056982512327213. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -25.472944182933528 (T) = (0 -23.645774694088463) / Math.Sqrt((0.1302430376724827 / (23)) + (0.4615801379454354 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.1826468503281096 = (19.993943828225863 - 23.645774694088463) / 19.993943828225863 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 10.81 ns | 13.54 ns | 1.25 | 0.05 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<UInt64>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.536802932034956 > 11.35145753448431. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -8.574082760730478 (T) = (0 -13.506688811973898) / Math.Sqrt((1.7762903502862928 / (23)) + (0.10097248716469229 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.21965381121373115 = (11.074198832316851 - 13.506688811973898) / 11.074198832316851 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 6.14 ns | 8.24 ns | 1.34 | 0.20 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.243436857714752 > 6.447982991414564. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -8.389320644851091 (T) = (0 -8.063495790669068) / Math.Sqrt((0.03524392530140337 / (24)) + (1.7697574079138925 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.33647761884757654 = (6.033393808436607 - 8.063495790669068) / 6.033393808436607 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
### Run Information Architecture | x64 -- | -- OS | ubuntu 18.04 Baseline | [a8549002b2df6bf64c7c882d8f117630beba762d](https://github.com/dotnet/runtime/commit/a8549002b2df6bf64c7c882d8f117630beba762d) Compare | [8a2b72e3140b14a2ec915345b2747eecdecff943](https://github.com/dotnet/runtime/commit/8a2b72e3140b14a2ec915345b2747eecdecff943) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8549002b2df6bf64c7c882d8f117630beba762d...8a2b72e3140b14a2ec915345b2747eecdecff943) ### Regressions in System.Numerics.Tests.Perf_VectorOf<Int16> Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [GetHashCodeBenchmark - Duration of single invocation]() | 37.85 ns | 51.66 ns | 1.37 | 0.07 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Int16>*' ```
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 51.66120195005749 > 39.72074575315014. IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -39.559854435578245 (T) = (0 -51.881876605118066) / Math.Sqrt((0.8104291630112155 / (23)) + (2.8603275361064218 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.37413871248681874 = (37.75592386246504 - 51.881876605118066) / 37.75592386246504 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Author: performanceautofiler[bot]
Assignees: EgorBo
Labels: `os-linux`, `tenet-performance`, `tenet-performance-benchmarks`, `arch-x64`, `area-CodeGen-coreclr`
Milestone: -
EgorBo commented 1 year ago

So it is no longer inlined:

multiplier in methods of struct increased to 3.
Inline candidate is generic and caller is not.  Multiplier increased to 5.
Inline has 2 foldable BOX ops.  Multiplier increased to 8.
Inline candidate callsite is in a loop.  Multiplier increased to 11.
calleeNativeSizeEstimate=1251
callsiteNativeSizeEstimate=85
benefit multiplier=11
threshold=935
Native estimate for function size exceeds threshold for inlining 125.1 > 93.5 (multiplier = 11)
EgorBo commented 1 year ago

So I assume Roslyn has fixed some IL bug or something like that (e.g. https://github.com/dotnet/roslyn/pull/64789) so this is not actionable

jakobbotsch commented 1 year ago

The codegen difference in the screenshot above looks like the result of https://github.com/dotnet/roslyn/pull/65642, related issue in dotnet/runtime is #73606.

EgorBo commented 1 year ago

Performance impact of that change, FYI @jaredpar @AlekseyTs

jaredpar commented 1 year ago

Is there a suggestion on how we can emit this code better? The runtime team asked us to fix a bug which to our knowledge requires the new codegen. Until we properly spill these values codegen can't fix their bug.

EgorBo commented 1 year ago

Is there a suggestion on how we can emit this code better? The runtime team asked us to fix a bug which to our knowledge requires the new codegen. Until we properly spill these values codegen can't fix their bug.

I just wanted to let you know that we also try to track perf issues after Roslyn updates 🙂 - these regressions don't look bad to me at all honestly, the HashCode.Combine is perf critical, but the big performance difference is purely because of inlining. So I assume we can close this. Unless somebody thinks it should be improved.

jakobbotsch commented 1 year ago

Is there a suggestion on how we can emit this code better? The runtime team asked us to fix a bug which to our knowledge requires the new codegen. Until we properly spill these values codegen can't fix their bug.

A copy is only needed when the receiver could actually be changed between loading its address and the constrained call. It is not the case in the above screenshot (and probably in most cases). However, it would of course require some analysis to determine whether it was safe to omit the copy.

AlekseyTs commented 1 year ago

A copy is only needed when the receiver could actually be changed between loading its address and the constrained call. It is not the case in the above screenshot (and probably in most cases). However, it would of course require some analysis to determine whether it was safe to omit the copy.

We do detect some arguments "safe" for reordering, but it was not our goal to be able to accurately detect "safety" for non-trivial scenarios. I assume that the argument in the scenario is not trivial, perhaps another method call or something similar.

jakobbotsch commented 1 year ago

@AlekseyTs maybe I am missing something, but there are no args in the above screenshot -- the IL emitted before is

ldarga.s value1
constrained. !!T1
callvirt instance int32 System.Object::GetHashCode()
AlekseyTs commented 1 year ago

maybe I am missing something, but there are no args in the above screenshot

What C# code looks like? Perhaps the IL is different due to a different reason. Due to https://github.com/dotnet/roslyn/pull/66250, for example.

jakobbotsch commented 1 year ago

What C# code looks like? Perhaps the IL is different due to a different reason

https://github.com/dotnet/runtime/blob/63f46d2bcbc4e65d1db566f79ba1aff64de505c3/src/libraries/System.Private.CoreLib/src/System/HashCode.cs#L87

So actually https://github.com/dotnet/roslyn/pull/66250 might be the cause after all?

AlekseyTs commented 1 year ago

I think so.

jakobbotsch commented 1 year ago

We can change the runtime implementation here to avoid the conditional access to fix this regression. However, it kind of puts us down a path of never using conditional access on generically typed values that may be structs.

tannergooding commented 1 year ago

@jakobbotsch what about putting some level of “trust” in the C# language features such as “readonly members” as a way to determine “this method mutates vs not”?

That is, rather than doing the complex analysis on the JIT side to determine if a non inlined method does mutate x, and would therefore require a copy to be preserved, we can check for the IsReadOnlyAttribute and use that as a much cheaper way of determining the same thing.

There are of course unsafe scenarios where that may change the observed output, but it will be correct in the vast majority of cases and match how C# will itself emit code in other scenarios involving the same (minus generics because no “readonly” concept exists for reference types, hence the general regression here)

That should allow us to much more trivially fix this case and improve codegen in a number of other places while also taking advantage of C# metadata that already tracks the info we care about

jakobbotsch commented 1 year ago

That should allow us to much more trivially fix this case and improve codegen in a number of other places while also taking advantage of C# metadata that already tracks the info we care about

We likely have all information we need to eliminate this copy in the vast majority of cases even today (e.g. in the above case we can see that the copy is the last use, so nothing fundamentally prevents us from removing it). But actually doing so requires #8887.

jakobbotsch commented 1 year ago

Once we have #8887 I think it makes good sense to experiment with using IsReadOnlyAttribute too as an input to whether reverse copy propagation can be done safely.