search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/arm64: 22 Improvements on 11/30/2022 10:53:28 AM #10536

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueTrue(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 43.90 μs 29.53 μs 0.67 0.23 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 29.533966194968556 < 31.78006395947829. IsChangePoint: Marked as a change because one of 11/6/2022 12:38:35 PM, 11/7/2022 10:11:20 AM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 17.6988826717032 (T) = (0 -29991.192095113885) / Math.Sqrt((15948704.788641788 / (299)) + (897237.5308044587 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.15092517460192909 = (35322.20152806102 - 29991.192095113885) / 35322.20152806102 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in IfStatements.IfStatements

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AndAnd - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/IfStatements.IfStatements.AndAnd.html>) 57.70 μs 41.39 μs 0.72 0.01 False
[AndAndAnd - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/IfStatements.IfStatements.AndAndAnd.html>) 51.48 μs 46.22 μs 0.90 0.01 False
[And - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/IfStatements.IfStatements.And.html>) 71.11 μs 35.87 μs 0.50 0.01 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'IfStatements.IfStatements*'
### Payloads [Baseline]() [Compare]() ### Histogram #### IfStatements.IfStatements.AndAnd ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 41.390603741496605 < 54.803751530123485. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 45.420183722956196 (T) = (0 -41386.24679734355) / Math.Sqrt((19703207.601000212 / (299)) + (76.76389689866416 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.21980573085968622 = (53046.07895024214 - 41386.24679734355) / 53046.07895024214 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### IfStatements.IfStatements.AndAndAnd ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 46.22093327345732 < 48.922010602678576. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 43.08123025691668 (T) = (0 -46219.71951030754) / Math.Sqrt((1962354.4111437318 / (299)) + (171.17328057150252 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07024543255094397 = (49711.74235488771 - 46219.71951030754) / 49711.74235488771 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### IfStatements.IfStatements.And ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 35.8680167166549 < 67.461488377462. IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 51.61951795131342 (T) = (0 -35886.673251446446) / Math.Sqrt((61645490.99680327 / (299)) + (267.0801112386483 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.3950912408839795 = (59325.76229163751 - 35886.673251446446) / 59325.76229163751 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 262.52 ns 227.33 ns 0.87 0.00 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 261.58 ns 227.21 ns 0.87 0.00 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 227.33248634369124 < 248.81803064370914. IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 735.2070423712194 (T) = (0 -226.97055361357766) / Math.Sqrt((0.23168189002013354 / (299)) + (0.03690057375317679 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.13477216378863885 = (262.32460874979574 - 226.97055361357766) / 262.32460874979574 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 227.2072612216843 < 248.87509023746438. IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 423.8358744676547 (T) = (0 -226.77869512772975) / Math.Sqrt((0.35592060396304587 / (299)) + (0.14105948117765776 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.13578745198809292 = (262.41078731086094 - 226.77869512772975) / 262.41078731086094 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).SortedSet(Size%3a%20512).html>) 33.65 μs 31.44 μs 0.93 0.11 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 31.437826063524 < 31.956215344982073. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 11/18/2022 7:31:41 AM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 11.767690578970852 (T) = (0 -31312.104809280612) / Math.Sqrt((2569914.426581275 / (299)) + (1290094.9502864215 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0857905384819609 = (34250.471174611404 - 31312.104809280612) / 34250.471174611404 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.TryParse(value%3a%20%22-1.7976931348623157e%2b308%22).html>) 162.20 ns 152.78 ns 0.94 0.01 False
[TryParse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.TryParse(value%3a%20%221.7976931348623157e%2b308%22).html>) 155.39 ns 143.37 ns 0.92 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.Parse(value%3a%20%22-1.7976931348623157e%2b308%22).html>) 161.92 ns 153.33 ns 0.95 0.00 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 152.78272423617253 < 154.1032980639742. IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/5/2022 3:01:26 AM, 11/7/2022 4:10:19 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 53.769838937773045 (T) = (0 -152.9158352250743) / Math.Sqrt((6.289405058720941 / (299)) + (0.18079835565101507 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05609875719034734 = (162.00406174898038 - 152.9158352250743) / 162.00406174898038 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 143.3693732957847 < 147.74431853883402. IsChangePoint: Marked as a change because one of 11/5/2022 3:01:26 AM, 11/7/2022 4:10:19 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 86.0988828615962 (T) = (0 -143.47390273804055) / Math.Sqrt((4.81560535698414 / (299)) + (0.1638741752935399 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08330787560837681 = (156.51263812619663 - 143.47390273804055) / 156.51263812619663 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 153.33422143129098 < 153.6669921859692. IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/1/2022 8:41:52 AM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 45.652966221658325 (T) = (0 -153.62167065600542) / Math.Sqrt((2.0269556124520323 / (299)) + (0.6239451025719243 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.051055305811851255 = (161.88685346666432 - 153.62167065600542) / 161.88685346666432 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.Parse(value%3a%20%22-3.4028235E%2b38%22).html>) 124.35 ns 116.55 ns 0.94 0.00 False
[TryParse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.TryParse(value%3a%20%22-3.4028235E%2b38%22).html>) 123.85 ns 115.31 ns 0.93 0.01 False
[TryParse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.TryParse(value%3a%20%223.4028235E%2b38%22).html>) 117.95 ns 104.60 ns 0.89 0.00 False
[Parse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.Parse(value%3a%20%223.4028235E%2b38%22).html>) 118.07 ns 104.64 ns 0.89 0.00 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Single.Parse(value: "-3.4028235E+38") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 116.55286002322224 < 118.0839244070994. IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 37.456913252970665 (T) = (0 -116.52397440716555) / Math.Sqrt((3.0281182518058354 / (198)) + (0.5772967868808929 / (24))) is greater than 1.970805592384696 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (198) + (24) - 2, .975) and 0.05994187229104325 = (123.95400983463603 - 116.52397440716555) / 123.95400983463603 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 115.31241427418152 < 117.96823681424031. IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 52.82406069305587 (T) = (0 -116.01688702082589) / Math.Sqrt((3.890671876376401 / (299)) + (0.09703283099345228 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.056124826621696454 = (122.91549803728816 - 116.01688702082589) / 122.91549803728816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Single.TryParse(value: "3.4028235E+38") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 104.60218486351098 < 111.8284709248595. IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 66.375087954622 (T) = (0 -104.36770368536251) / Math.Sqrt((2.0594874079147694 / (299)) + (0.6447695664133842 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10461781720364112 = (116.56218505422211 - 104.36770368536251) / 116.56218505422211 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 104.63772486544954 < 112.02605834876927. IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 43.55588504944838 (T) = (0 -104.8843987216671) / Math.Sqrt((1.7558152183637488 / (299)) + (1.7717349129538018 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.10493165888585948 = (117.180324567297 - 104.8843987216671) / 117.180324567297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).SortedDictionary(Size%3a%20512).html>) 45.57 μs 43.23 μs 0.95 0.09 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 33.24 μs 29.96 μs 0.90 0.18 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyTrue<Int32, Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 43.23347844317284 < 43.39832244018006. IsChangePoint: Marked as a change because one of 9/22/2022 11:37:30 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 10.173651230910679 (T) = (0 -43388.737908041956) / Math.Sqrt((1435873.4740669765 / (299)) + (1138934.4811111153 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05087440087801577 = (45714.43226078819 - 43388.737908041956) / 45714.43226078819 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 29.962862284069107 < 31.51248013935793. IsChangePoint: Marked as a change because one of 9/20/2022 3:06:45 PM, 9/21/2022 4:00:51 AM, 10/31/2022 10:23:25 PM, 11/1/2022 1:15:18 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 7.337798769502817 (T) = (0 -30694.58505849182) / Math.Sqrt((9368771.78508014 / (299)) + (6468538.451600095 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1159239289555685 = (34719.39357235378 - 30694.58505849182) / 34719.39357235378 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrue(Int32).ImmutableSortedSet(Size%3a%20512).html>) 30.32 μs 27.75 μs 0.92 0.19 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 27.750222419928825 < 28.76831916184975. IsChangePoint: Marked as a change because one of 10/10/2022 11:56:05 PM, 10/11/2022 10:45:23 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 13.837872183653896 (T) = (0 -28479.008537190737) / Math.Sqrt((6141953.968756738 / (299)) + (733362.2134318674 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.0989666792744481 = (31607.05368171976 - 28479.008537190737) / 31607.05368171976 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HeapSort - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Guid%2c%20Guid).HeapSort(Size%3a%20100).html>) 8.05 μs 7.07 μs 0.88 0.04 False
[HeapSort - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Guid%2c%20Guid).HeapSort(Size%3a%201000).html>) 157.77 μs 135.59 μs 0.86 0.02 False
[Dequeue_And_Enqueue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Guid%2c%20Guid).Dequeue_And_Enqueue(Size%3a%20100).html>) 20.13 μs 19.02 μs 0.94 0.40 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.068287363129923 < 7.740348158215607. IsChangePoint: Marked as a change because one of 10/25/2022 5:34:05 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 28.419258294892373 (T) = (0 -7125.660877660739) / Math.Sqrt((106931.68496512793 / (299)) + (6047.485890825918 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08964466365404386 = (7827.340152980465 - 7125.660877660739) / 7827.340152980465 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 135.59103807471263 < 149.95049375. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 11/18/2022 7:31:41 AM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 29.92539238437066 (T) = (0 -135880.48307258677) / Math.Sqrt((67930264.53757167 / (299)) + (756106.1899988991 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.1007318466194701 = (151101.18440398973 - 135880.48307258677) / 151101.18440398973 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 19.01658653846154 < 20.38100523422255. IsChangePoint: Marked as a change because one of 11/29/2022 7:47:47 PM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 20.536754504412517 (T) = (0 -19180.754558711662) / Math.Sqrt((695910.672804359 / (299)) + (101257.06143102661 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.07877114531114546 = (20820.83562741852 - 19180.754558711662) / 20820.83562741852 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).ImmutableList(Size%3a%20512).html>) 122.65 μs 113.95 μs 0.93 0.12 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 113.94571256038645 < 117.69822263428007. IsChangePoint: Marked as a change because one of 10/25/2022 2:56:59 AM, 10/25/2022 10:16:52 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 8.904084862948022 (T) = (0 -115827.52185835905) / Math.Sqrt((43817050.32821752 / (299)) + (23970488.149504434 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.07601592929621694 = (125356.62197091253 - 115827.52185835905) / 125356.62197091253 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline d4d12c5563621db5ed31abaeb0143bf30c2d53d3
Compare 8158cb8be51bd6269c75e4010e2e25ada78578f0
Diff Diff

Improvements in SciMark2.kernel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[benchmarkLU - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/SciMark2.kernel.benchmarkLU.html>) 687.07 ms 584.38 ms 0.85 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SciMark2.kernel*'
### Payloads [Baseline]() [Compare]() ### Histogram #### SciMark2.kernel.benchmarkLU ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 584.3848 < 652.575976. IsChangePoint: Marked as a change because one of 10/31/2022 1:09:40 PM, 11/30/2022 2:05:04 AM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM. IsImprovementStdDev: Marked as improvement because 12.940830759633016 (T) = (0 -584663114.9191087) / Math.Sqrt((2402227143662805 / (299)) + (194702695095.03372 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05906198935306192 = (621361990.1667337 - 584663114.9191087) / 621361990.1667337 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)