search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 14 Improvements on 11/15/2022 10:02:59 PM #9976

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateFromScalarXYBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.CreateFromScalarXYBenchmark.html>) 20.10 ns 16.80 ns 0.84 0.39 False
[UnitXBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.UnitXBenchmark.html>) 19.60 ns 17.50 ns 0.89 0.34 False
[UnitYBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.UnitYBenchmark.html>) 19.75 ns 16.97 ns 0.86 0.38 False
[CreateFromScalar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.CreateFromScalar.html>) 18.40 ns 16.43 ns 0.89 0.40 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Vector2.CreateFromScalarXYBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 16.796087664532187 < 18.995665350286984. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 7.324815433919425 (T) = (0 -17.253166749745326) / Math.Sqrt((4.899673487930778 / (299)) + (0.7263571383176524 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08507239352561077 = (18.857411917243617 - 17.253166749745326) / 18.857411917243617 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector2.UnitXBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 17.495542777847938 < 18.27586278886116. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 8.1031657090824 (T) = (0 -17.13316980526051) / Math.Sqrt((4.078549583868242 / (299)) + (0.8466399482118598 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09469817898754755 = (18.925367659262495 - 17.13316980526051) / 18.925367659262495 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector2.UnitYBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 16.968592324750414 < 18.577099128279695. IsChangePoint: Marked as a change because one of 10/19/2022 1:55:59 PM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 5.394290275247977 (T) = (0 -17.52596139529805) / Math.Sqrt((2.6351775207308745 / (299)) + (0.9652366169624401 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06380522482855557 = (18.720422138746223 - 17.52596139529805) / 18.720422138746223 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector2.CreateFromScalar ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 16.431868472630054 < 17.426093253660422. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 6.722647425123183 (T) = (0 -15.80891259469089) / Math.Sqrt((4.0197239104895495 / (299)) + (0.7764695074621232 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.08232300757193393 = (17.22709921370302 - 15.80891259469089) / 17.22709921370302 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in System.Memory.Constructors<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MemoryMarshalCreateReadOnlySpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(Byte).MemoryMarshalCreateReadOnlySpan.html>) 19.37 ns 17.19 ns 0.89 0.26 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Constructors<Byte>.MemoryMarshalCreateReadOnlySpan ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 17.187669048821665 < 18.420203709343582. IsChangePoint: Marked as a change because one of 9/25/2022 5:06:47 PM, 9/26/2022 2:03:14 AM, 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 6.40471282139619 (T) = (0 -18.09276817673842) / Math.Sqrt((2.448851898637405 / (299)) + (0.40038981590790246 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.052199995587193924 = (19.0892256726117 - 18.09276817673842) / 19.0892256726117 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CrossBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.CrossBenchmark.html>) 77.93 ns 66.25 ns 0.85 0.13 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Vector3.CrossBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 66.25127372136053 < 80.82137761135793. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 18.504300201810313 (T) = (0 -68.5521289666358) / Math.Sqrt((23.424422799884912 / (299)) + (13.372026632302505 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.17982260369291655 = (83.58207538429788 - 68.5521289666358) / 83.58207538429788 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsAllBenchmark.html>) 108.28 ns 97.51 ns 0.90 0.18 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 97.50836621777867 < 99.62614871810203. IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 6.315694867295252 (T) = (0 -97.3722681647104) / Math.Sqrt((23.60279311329894 / (28)) + (0.5376297475073795 / (25))) is greater than 2.0075837703147283 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (25) - 2, .975) and 0.05687582457272504 = (103.24437725350076 - 97.3722681647104) / 103.24437725350076 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in StoreBlock.AnyLocation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CopyBlock16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/StoreBlock.AnyLocation.CopyBlock16.html>) 22.16 ns 20.75 ns 0.94 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'StoreBlock.AnyLocation*'
### Payloads [Baseline]() [Compare]() ### Histogram #### StoreBlock.AnyLocation.CopyBlock16 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 20.751004197190074 < 21.055053578735063. IsChangePoint: Marked as a change because one of 8/26/2022 11:36:04 AM, 9/19/2022 6:07:30 AM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 17.146808754701308 (T) = (0 -20.869912048690313) / Math.Sqrt((0.6103755790733023 / (299)) + (0.08059914867256285 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.0576567330762018 = (22.146825664514395 - 20.869912048690313) / 22.146825664514395 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).SortedList(Size%3a%20512).html>) 80.45 μs 73.33 μs 0.91 0.08 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 73.33027326458752 < 73.96191619649424. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 26.885345827945425 (T) = (0 -73204.33925408739) / Math.Sqrt((5626040.674998625 / (299)) + (644821.6982599699 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07364133395482182 = (79023.7539058303 - 73204.33925408739) / 79023.7539058303 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in System.Tests.Perf_TimeSpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FromSeconds - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_TimeSpan.FromSeconds.html>) 118.18 ns 109.87 ns 0.93 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_TimeSpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_TimeSpan.FromSeconds ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 109.86860621310265 < 112.05978406717105. IsChangePoint: Marked as a change because one of 10/19/2022 4:51:05 PM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 14.585528693013456 (T) = (0 -111.15786617279109) / Math.Sqrt((15.886068244370602 / (299)) + (3.786083801896247 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.05602327528152603 = (117.75488024447017 - 111.15786617279109) / 117.75488024447017 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IdentityBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.IdentityBenchmark.html>) 17.07 ns 13.77 ns 0.81 0.44 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Matrix3x2.IdentityBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.772686446594504 < 15.979209914686788. IsChangePoint: Marked as a change because one of 9/30/2022 9:31:11 PM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 5.0662140747869175 (T) = (0 -15.02044110081482) / Math.Sqrt((4.365517150633143 / (299)) + (1.1404008180166072 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07872292850766899 = (16.30393457690633 - 15.02044110081482) / 16.30393457690633 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).EqualsAllBenchmark.html>) 102.68 ns 95.31 ns 0.93 0.12 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 95.30788485831138 < 95.35480723633012. IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 8.751926897805975 (T) = (0 -94.33082693281739) / Math.Sqrt((8.38886941085687 / (30)) + (1.0837501238881522 / (25))) is greater than 2.0057459953168344 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (25) - 2, .975) and 0.05008656003001263 = (99.30465552292615 - 94.33082693281739) / 99.30465552292615 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).EqualsAllBenchmark.html>) 103.55 ns 93.67 ns 0.90 0.25 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 93.67064518020932 < 95.75129900249874. IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 5.781342111031923 (T) = (0 -94.47989471436102) / Math.Sqrt((25.10425707959289 / (28)) + (1.2200662554437012 / (25))) is greater than 2.0075837703147283 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (25) - 2, .975) and 0.056155667665502965 = (100.10114112851132 - 94.47989471436102) / 100.10114112851132 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EmptyStaticFunction5Arg - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg.html>) 36.31 ms 29.96 ms 0.82 0.10 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 29.955480798076923 < 35.085576437619046. IsChangePoint: Marked as a change because one of 9/23/2022 2:49:56 PM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/11/2022 3:15:52 PM, 11/15/2022 8:48:03 PM, 11/22/2022 6:55:51 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 6:55:51 AM. IsImprovementStdDev: Marked as improvement because 22.165386942399877 (T) = (0 -30154529.217974357) / Math.Sqrt((2492174807196.159 / (299)) + (105265122795.62495 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.07606915657778247 = (32637214.606109165 - 30154529.217974357) / 32637214.606109165 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
lewing commented 1 year ago

looks like https://github.com/dotnet/runtime/pull/76477 cc @kg