dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 72 Improvements on 10/23/2022 1:14:54 PM #9236

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare e232f7918900c30aa15cf2fca8056296a6bef1e5
Diff Diff

Improvements in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SameIndex1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.SameIndex1(length%3a%201024).html>) 22.49 μs 18.67 μs 0.83 0.12 False
[CoveredIndex1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.CoveredIndex1(length%3a%201024).html>) 25.57 μs 21.75 μs 0.85 0.12 False
[SameIndex2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.SameIndex2(length%3a%201024).html>) 25.91 μs 21.74 μs 0.84 0.05 False
[ReadOnlyIndexer1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.ReadOnlyIndexer1(length%3a%201024).html>) 13.27 μs 11.29 μs 0.85 0.05 False
[Indexer4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.Indexer4(length%3a%201024).html>) 131.37 μs 111.90 μs 0.85 0.10 False
[Indexer1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.Indexer1(length%3a%201024).html>) 13.27 μs 11.28 μs 0.85 0.05 False
[CoveredIndex2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.CoveredIndex2(length%3a%201024).html>) 25.62 μs 21.55 μs 0.84 0.11 False
[WriteViaIndexer1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.WriteViaIndexer1(length%3a%201024).html>) 24.31 μs 20.50 μs 0.84 0.10 False
[CoveredIndex3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.CoveredIndex3(length%3a%201024).html>) 36.67 μs 30.56 μs 0.83 0.14 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Span.IndexerBench.SameIndex1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.66843579404467 < 21.363974391276457. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 61.36269824675997 (T) = (0 -18684.780310627506) / Math.Sqrt((3146913.328461795 / (299)) + (340.64797268907967 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2525179444187288 = (24996.961694414847 - 18684.780310627506) / 24996.961694414847 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.CoveredIndex1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.751784464534424 < 24.28950444401011. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 60.11656477594083 (T) = (0 -21753.554802545994) / Math.Sqrt((3368032.4019381464 / (299)) + (217.07614379267082 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22712342390370147 = (28146.220852520124 - 21753.554802545994) / 28146.220852520124 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.SameIndex2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.74483512650882 < 24.61377649478303. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 70.31377404040067 (T) = (0 -21765.72605821868) / Math.Sqrt((2507512.1791640874 / (299)) + (659.2529721687966 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22967536563149688 = (28255.264192689086 - 21765.72605821868) / 28255.264192689086 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.ReadOnlyIndexer1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.293232569997588 < 12.57327492038875. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 55.175454301647974 (T) = (0 -11310.169137732346) / Math.Sqrt((704571.4323538989 / (299)) + (4982.076222136535 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.21592773222401085 = (14424.906481915876 - 11310.169137732346) / 14424.906481915876 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.Indexer4(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 111.90188105867348 < 124.82942034340657. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 57.13258983054446 (T) = (0 -111938.38882568025) / Math.Sqrt((95847851.7486639 / (299)) + (1074.6172333295312 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2242481290860705 = (144296.64048866977 - 111938.38882568025) / 144296.64048866977 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.Indexer1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.283766702866506 < 12.579714865581408. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 66.90709097518882 (T) = (0 -11276.734744144873) / Math.Sqrt((684373.8153952889 / (299)) + (27.722400311401817 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.22127085299910226 = (14480.946022856228 - 11276.734744144873) / 14480.946022856228 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.CoveredIndex2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.553279133679254 < 24.352113165406937. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 57.48573485029748 (T) = (0 -21569.52502858697) / Math.Sqrt((4028300.9679123373 / (299)) + (92.3788401593988 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.23638374738595536 = (28246.550482325678 - 21569.52502858697) / 28246.550482325678 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.WriteViaIndexer1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 20.497727517488958 < 23.091603526538748. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 49.609399806446525 (T) = (0 -20297.30747918355) / Math.Sqrt((958816.8811812779 / (299)) + (45717.588327159596 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.20275004010124423 = (25459.151458296896 - 20297.30747918355) / 25459.151458296896 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.CoveredIndex3(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 30.558031767775606 < 34.90362324277091. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 55.97209010364292 (T) = (0 -30564.663579155356) / Math.Sqrt((9368335.175218407 / (299)) + (1509.5296529140496 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.24553921166111914 = (40511.93123826952 - 30564.663579155356) / 40511.93123826952 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare bbcff6bc422da3d9b1904281a86014236ce50f2e
Diff Diff

Improvements in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(Int32).Span(Size%3a%20512).html>) 7.04 μs 6.02 μs 0.86 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.IndexerSet<Int32>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.022314739865739 < 6.692420791721815. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 14.80597131557725 (T) = (0 -6143.517914762402) / Math.Sqrt((216538.57881169367 / (299)) + (63275.019270168654 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.21799423779551172 = (7856.103128247693 - 6143.517914762402) / 7856.103128247693 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare e232f7918900c30aa15cf2fca8056296a6bef1e5
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseUInt64Hex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value%3a%200).html>) 157.46 ns 146.23 ns 0.93 0.17 False
[TryParseSByte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value%3a%20127).html>) 220.91 ns 192.04 ns 0.87 0.21 False
[TryParseInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value%3a%20-9223372036854775808).html>) 582.74 ns 545.18 ns 0.94 0.09 False
[TryParseUInt32Hex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value%3a%20FFFFFFFFFFFFFFFF).html>) 349.15 ns 311.17 ns 0.89 0.11 False
[TryParseByte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value%3a%20255).html>) 192.40 ns 168.30 ns 0.87 0.19 False
[TryParseInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value%3a%209223372036854775807).html>) 565.06 ns 530.61 ns 0.94 0.07 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 0) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 146.23265385271716 < 151.5500832976479. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 43.25438147702853 (T) = (0 -146.16583209572283) / Math.Sqrt((88.75403118085093 / (299)) + (0.2596694709703565 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.147181553600911 = (171.39149922575945 - 146.16583209572283) / 171.39149922575945 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: 127) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 192.04279303645248 < 209.17648783727014. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 8.32451233347546 (T) = (0 -197.9682477986811) / Math.Sqrt((84.99961819016242 / (299)) + (71.3042776594162 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13820564661109172 = (229.71634360354471 - 197.9682477986811) / 229.71634360354471 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 545.177192381728 < 553.1323862719198. IsChangePoint: Marked as a change because one of 10/11/2022 5:13:36 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 52.7390048163061 (T) = (0 -545.4899957289873) / Math.Sqrt((563.8128613407517 / (299)) + (0.9072971251679636 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12125275577337669 = (620.7586986051576 - 545.4899957289873) / 620.7586986051576 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: FFFFFFFFFFFFFFFF) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 311.16627081488457 < 335.05660857570876. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 35.79385642467714 (T) = (0 -311.14437110127176) / Math.Sqrt((428.62503496347296 / (299)) + (15.141369443533694 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19549311846913334 = (386.75165899041974 - 311.14437110127176) / 386.75165899041974 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 255) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 168.29521539320538 < 183.0489854434753. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 65.8441480151179 (T) = (0 -168.77782697937923) / Math.Sqrt((48.941824432929266 / (299)) + (0.6614830518172201 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.16956265591385694 = (203.23968831761923 - 168.77782697937923) / 203.23968831761923 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 530.6095557350566 < 535.5728620422815. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/24/2022 9:01:44 AM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 52.24127951460963 (T) = (0 -528.163148836587) / Math.Sqrt((350.4653743064082 / (299)) + (2.6631404147320614 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.11823851498359444 = (598.9864127789164 - 528.163148836587) / 598.9864127789164 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare e232f7918900c30aa15cf2fca8056296a6bef1e5
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Guids

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteGuids - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 213.24 ms 191.15 ms 0.90 0.04 False
[WriteGuids - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 209.11 ms 189.07 ms 0.90 0.05 False
[WriteGuids - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted%3a%20False%2c%20SkipValidation%3a%20False).html>) 195.34 ms 174.47 ms 0.89 0.01 False
[WriteGuids - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 190.43 ms 172.80 ms 0.91 0.01 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Guids*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 191.14538453846154 < 201.31490728083332. IsChangePoint: Marked as a change because one of 9/19/2022 9:42:49 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 47.60450366288845 (T) = (0 -191446847.12823567) / Math.Sqrt((28078475437498.047 / (299)) + (1474347906138.0073 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12657046574114522 = (219189802.51874256 - 191446847.12823567) / 219189802.51874256 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 189.0700664 < 198.98332667119047. IsChangePoint: Marked as a change because one of 9/19/2022 6:07:30 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 23.917799514520702 (T) = (0 -189345522.6858608) / Math.Sqrt((115364745585300.44 / (299)) + (4725275538562.707 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12719104027265224 = (216938106.07192835 - 189345522.6858608) / 216938106.07192835 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 174.4681997333333 < 186.55295679333332. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 38.6799108060424 (T) = (0 -176043462.32307693) / Math.Sqrt((23214494315453.51 / (299)) + (2160368631320.4597 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12691457777902015 = (201633720.87378633 - 176043462.32307693) / 201633720.87378633 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 172.797667 < 183.907365. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 42.4842095142074 (T) = (0 -174141661.14603174) / Math.Sqrt((23910876394585.254 / (299)) + (1551946446544.1658 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12431754990028634 = (198863938.77849475 - 174141661.14603174) / 198863938.77849475 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare e232f7918900c30aa15cf2fca8056296a6bef1e5
Diff Diff

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).Span(Size%3a%20512).html>) 11.70 μs 10.73 μs 0.92 0.08 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.CreateAddAndClear<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.731538536641397 < 11.126323359930813. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 29.998574315531453 (T) = (0 -10734.029806148266) / Math.Sqrt((223341.14624077833 / (299)) + (1599.832538088753 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.0836501027687705 = (11713.898630404568 - 10734.029806148266) / 11713.898630404568 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ebf4ea7601f82cdc3f7cb9e7539a84e423d0d23a
Compare bbcff6bc422da3d9b1904281a86014236ce50f2e
Diff Diff

Improvements in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FixedTimeEquals_256Bit_Equal - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal.html>) 700.05 ns 581.92 ns 0.83 0.10 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 581.9154799155707 < 668.0363925019811. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 16.227996707668204 (T) = (0 -590.2209076241182) / Math.Sqrt((260.35678492835103 / (299)) + (280.62212818569907 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.17189223508462545 = (712.7344201203496 - 590.2209076241182) / 712.7344201203496 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
lewing commented 1 year ago

this is https://github.com/dotnet/runtime/pull/77331