search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 5/9/2022 3:10:19 PM #5245

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ClassVirtualMethod - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.LowLevelPerf.ClassVirtualMethod.html>) 130.55 μs 108.70 μs 0.83 0.14 False
[IntegerFormatting - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.LowLevelPerf.IntegerFormatting.html>) 1.88 ms 1.65 ms 0.88 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.ClassVirtualMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 108.69707034350198 < 124.01375627610365. IsChangePoint: Marked as a change because one of 3/16/2022 10:57:58 PM, 3/21/2022 1:28:14 PM, 4/20/2022 8:23:28 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 36.63992501064552 (T) = (0 -108738.15438542939) / Math.Sqrt((70839833.41667259 / (299)) + (1739.550837838994 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14099120929915404 = (126585.61304909624 - 108738.15438542939) / 126585.61304909624 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.IntegerFormatting ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.6523709482142859 < 1.7680112245602486. IsChangePoint: Marked as a change because one of 3/23/2022 3:13:36 PM, 4/16/2022 5:36:25 PM, 5/5/2022 12:13:02 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 18.95329341767364 (T) = (0 -1711768.3930121446) / Math.Sqrt((1549324408.4594517 / (299)) + (917514868.1389276 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06295670680645749 = (1826776.2070824467 - 1711768.3930121446) / 1826776.2070824467 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Singleton - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode%3a%20%22Dynamic%22).html>) 2.79 μs 2.46 μs 0.88 0.05 False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22Expressions%22).html>) 2.98 μs 2.79 μs 0.94 0.06 False
[Singleton - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode%3a%20%22ILEmit%22).html>) 2.84 μs 2.46 μs 0.87 0.04 False
[Transient - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode%3a%20%22Expressions%22).html>) 2.88 μs 2.56 μs 0.89 0.04 False
[Transient - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode%3a%20%22Runtime%22).html>) 2.83 μs 2.59 μs 0.92 0.03 False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22Dynamic%22).html>) 2.96 μs 2.78 μs 0.94 0.06 False
[Singleton - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode%3a%20%22Runtime%22).html>) 2.87 μs 2.44 μs 0.85 0.05 False
[Singleton - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode%3a%20%22Expressions%22).html>) 2.87 μs 2.50 μs 0.87 0.03 False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22Runtime%22).html>) 2.99 μs 2.71 μs 0.90 0.04 False
[Transient - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode%3a%20%22ILEmit%22).html>) 2.86 μs 2.57 μs 0.90 0.04 False
[Transient - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode%3a%20%22Dynamic%22).html>) 2.93 μs 2.61 μs 0.89 0.06 False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22ILEmit%22).html>) 3.12 μs 2.66 μs 0.85 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.457675834912364 < 2.7118903801153396. IsChangePoint: Marked as a change because one of 3/17/2022 11:30:59 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 24.926811869083277 (T) = (0 -2507.9047026982134) / Math.Sqrt((3078.15577552649 / (299)) + (947.0091479193768 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12315795734029308 = (2860.155627450342 - 2507.9047026982134) / 2860.155627450342 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.78794880713649 < 2.871688301752536. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 25.649698420141757 (T) = (0 -2746.7826167376647) / Math.Sqrt((4643.713670895385 / (299)) + (654.5456945164259 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1015293612510744 = (3057.1757142363817 - 2746.7826167376647) / 3057.1757142363817 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.45825865099926 < 2.6925535366984708. IsChangePoint: Marked as a change because one of 3/22/2022 2:38:09 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 24.49502386518641 (T) = (0 -2508.9286523842266) / Math.Sqrt((3186.546489231141 / (299)) + (1002.4069297381589 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12423929165512893 = (2864.856379690673 - 2508.9286523842266) / 2864.856379690673 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.56290684678643 < 2.843175178159489. IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 80.51808366938361 (T) = (0 -2558.641739508864) / Math.Sqrt((2699.462960290876 / (299)) + (58.10449016436288 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12510924825050437 = (2924.5271302644546 - 2558.641739508864) / 2924.5271302644546 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.5926628051468565 < 2.743058907132944. IsChangePoint: Marked as a change because one of 3/11/2022 4:59:21 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 37.46014516063466 (T) = (0 -2579.900601116417) / Math.Sqrt((2703.9061373534523 / (299)) + (372.9285268885608 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11721889541451783 = (2922.469214299543 - 2579.900601116417) / 2922.469214299543 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.7752769106060993 < 2.8925902270867576. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 17.690616459329263 (T) = (0 -2740.4471990632796) / Math.Sqrt((4361.540557610913 / (299)) + (1601.029380974148 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10563862222772329 = (3064.1385766113162 - 2740.4471990632796) / 3064.1385766113162 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.4407872654163523 < 2.7354420242472863. IsChangePoint: Marked as a change because one of 3/18/2022 9:29:53 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 20.887743125628248 (T) = (0 -2501.1316091018402) / Math.Sqrt((3370.7105362633483 / (299)) + (1461.2398263688428 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12701532919188036 = (2865.034968811708 - 2501.1316091018402) / 2865.034968811708 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.4992384202656925 < 2.7220837721862536. IsChangePoint: Marked as a change because one of 3/21/2022 3:20:48 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 27.03831840550127 (T) = (0 -2496.210565903757) / Math.Sqrt((3008.294400293275 / (299)) + (850.138669199064 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1269113079719806 = (2859.057262676869 - 2496.210565903757) / 2859.057262676869 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.7067866542611587 < 2.919779521899807. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 16.28676544324097 (T) = (0 -2746.4532257589412) / Math.Sqrt((4062.300966405091 / (299)) + (1818.2526782776547 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10328219714691887 = (3062.784319682925 - 2746.4532257589412) / 3062.784319682925 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.5697747408729508 < 2.7806004223866663. IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 60.81014664735382 (T) = (0 -2564.448619620164) / Math.Sqrt((4200.6037427663805 / (299)) + (102.35827501108956 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12228522260663525 = (2921.733444247182 - 2564.448619620164) / 2921.733444247182 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.605150973008211 < 2.772370256506149. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/18/2022 1:59:24 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 40.29067599838805 (T) = (0 -2592.5959164525075) / Math.Sqrt((3741.555897717276 / (299)) + (274.56061972406303 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11316803055222635 = (2923.4353358583876 - 2592.5959164525075) / 2923.4353358583876 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.662394927613598 < 2.931102141704977. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 19.540713080285844 (T) = (0 -2708.540142968352) / Math.Sqrt((3916.1858698004894 / (299)) + (1537.8100052491016 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11440906248124874 = (3058.455126649263 - 2708.540142968352) / 3058.455126649263 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ThrowAndCatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Exceptions.Handling.ThrowAndCatch(kind%3a%20ReflectionSoftware).html>) 33.63 μs 29.52 μs 0.88 0.00 False
[ThrowAndCatchDeepRecursive - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Exceptions.Handling.ThrowAndCatchDeepRecursive(kind%3a%20ReflectionSoftware).html>) 43.64 μs 39.47 μs 0.90 0.01 False
[ThrowAndCatch_ManyCatchBlocks - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind%3a%20ReflectionHardware).html>) 32.36 μs 28.81 μs 0.89 0.01 False
[ThrowAndCatch_ManyCatchBlocks - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind%3a%20ReflectionSoftware).html>) 33.74 μs 29.90 μs 0.89 0.01 False
[ThrowAndCatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Exceptions.Handling.ThrowAndCatch(kind%3a%20ReflectionHardware).html>) 31.42 μs 28.51 μs 0.91 0.00 False
[ThrowAndCatchDeep - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Exceptions.Handling.ThrowAndCatchDeep(kind%3a%20ReflectionSoftware).html>) 42.59 μs 38.87 μs 0.91 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Exceptions.Handling*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Exceptions.Handling.ThrowAndCatch(kind: ReflectionSoftware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 29.51773797753565 < 31.979525132936576. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 14.93387148383593 (T) = (0 -29835.42838051273) / Math.Sqrt((1105771.3305545242 / (299)) + (120620.73454133287 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07705700729163172 = (32326.40435674247 - 29835.42838051273) / 32326.40435674247 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: ReflectionSoftware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 39.469092464826836 < 41.21535009685424. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 13.162228050064165 (T) = (0 -39622.6814946555) / Math.Sqrt((1194717.366412764 / (299)) + (171209.07254730596 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06099554104450533 = (42196.47853294536 - 39622.6814946555) / 42196.47853294536 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: ReflectionHardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 28.805566143644956 < 30.4764052538493. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 21.110061672874764 (T) = (0 -28999.465256932577) / Math.Sqrt((967878.9077697936 / (299)) + (30397.686523534918 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.0656504061524937 = (31037.06091155591 - 28999.465256932577) / 31037.06091155591 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: ReflectionSoftware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 29.904791492620443 < 32.06108875584953. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 21.555056038451077 (T) = (0 -30144.995004132506) / Math.Sqrt((1025167.9165101836 / (299)) + (48899.70951812094 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07593868568324362 = (32622.288734618738 - 30144.995004132506) / 32622.288734618738 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatch(kind: ReflectionHardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 28.50818678908151 < 30.233365972211327. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 20.196544133801204 (T) = (0 -28744.650720158075) / Math.Sqrt((906026.6048358857 / (299)) + (35669.28450443956 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06615008779008223 = (30780.80357916941 - 28744.650720158075) / 30780.80357916941 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchDeep(kind: ReflectionSoftware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 38.873671027561144 < 40.39234718646885. IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 16.521618819425882 (T) = (0 -38864.729832598125) / Math.Sqrt((1010634.3586232482 / (299)) + (93773.61682732476 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.059484062943413886 = (41322.7764690816 - 38864.729832598125) / 41322.7764690816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value%3a%209223372036854775807).html>) 22.90 ns 16.44 ns 0.72 0.08 False
[TryParseInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value%3a%20-9223372036854775808).html>) 22.84 ns 16.82 ns 0.74 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 16.442311752095176 < 21.762735142431065. IsChangePoint: Marked as a change because one of 3/5/2022 2:08:07 PM, 4/14/2022 8:50:18 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 15.386411985875512 (T) = (0 -16.388577188960635) / Math.Sqrt((8.864303188169348 / (299)) + (0.011575678199312666 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1437226038352413 = (19.13933178939978 - 16.388577188960635) / 19.13933178939978 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 16.816201258974036 < 22.11268545361661. IsChangePoint: Marked as a change because one of 3/5/2022 6:21:59 AM, 4/14/2022 8:50:18 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 14.759509053443765 (T) = (0 -16.918433458131116) / Math.Sqrt((8.902030439408303 / (299)) + (0.036627724542047685 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14385802737439343 = (19.761247548985192 - 16.918433458131116) / 19.761247548985192 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%22%2c%20value%3a%20%22x%22%2c%20comparisonType%3a%20InvariantCultureIgno.html>) 239.76 ns 206.16 ns 0.86 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 206.1591288128516 < 227.74231989634623. IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/15/2022 11:00:34 PM, 4/9/2022 5:43:54 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 26.902923428576766 (T) = (0 -206.77813176635271) / Math.Sqrt((228.36380410556407 / (299)) + (0.6914673490356407 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10997925819436982 = (232.32956497940873 - 206.77813176635271) / 232.32956497940873 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyTwoValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Byte).IndexOfAnyTwoValues(Size%3a%20512).html>) 7.64 ns 6.40 ns 0.84 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.404596817407116 < 7.274794285504554. IsChangePoint: Marked as a change because one of 3/21/2022 4:58:35 AM, 4/7/2022 3:14:09 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 11.74483203068309 (T) = (0 -6.470570254292866) / Math.Sqrt((0.29724314482861286 / (299)) + (0.01262677140890802 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09721388170642833 = (7.167334680027435 - 6.470570254292866) / 7.167334680027435 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NextBytes_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes_unseeded.html>) 169.66 ns 143.38 ns 0.85 0.05 False
[NextBytes_span_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Random.NextBytes_span_unseeded.html>) 169.86 ns 143.52 ns 0.84 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Random.NextBytes_unseeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 143.3833457408155 < 161.26569022280634. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/31/2022 1:45:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 22.998625811744972 (T) = (0 -143.36140481344188) / Math.Sqrt((157.94427380747956 / (299)) + (0.018311049194404048 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10474493355344629 = (160.1347037135148 - 143.36140481344188) / 160.1347037135148 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextBytes_span_unseeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 143.51504832685453 < 161.3672676390122. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/31/2022 1:45:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 22.44051960793765 (T) = (0 -143.398823457054) / Math.Sqrt((162.94877043354654 / (299)) + (0.03070729824366387 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1040827055601778 = (160.0581039645127 - 143.398823457054) / 160.0581039645127 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.TryAddGiventSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryAddGiventSize(Int32).Dictionary(Count%3a%20512).html>) 4.66 μs 3.27 μs 0.70 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddGiventSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddGiventSize<Int32>.Dictionary(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.268795280656304 < 4.424470863416372. IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 13.127953598660477 (T) = (0 -3278.9566708049338) / Math.Sqrt((400148.4800556268 / (299)) + (161.70580057757746 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.129090572176183 = (3764.980107056849 - 3278.9566708049338) / 3764.980107056849 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateFor(String).ImmutableSortedSet(Size%3a%20512).html>) 12.34 μs 10.67 μs 0.86 0.04 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateFor(String).ImmutableArray(Size%3a%20512).html>) 227.01 ns 120.49 ns 0.53 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.66538054513415 < 11.675786885031775. IsChangePoint: Marked as a change because one of 2/26/2022 10:27:55 AM, 3/17/2022 6:54:53 PM, 3/29/2022 3:04:55 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 35.54413990481685 (T) = (0 -10642.786265436822) / Math.Sqrt((354933.92499960854 / (299)) + (358.9421196795345 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10594170288386519 = (11903.906378103176 - 10642.786265436822) / 11903.906378103176 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateFor<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 120.49014924813193 < 215.8344044870088. IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 21.379201729914556 (T) = (0 -118.27011457491639) / Math.Sqrt((2850.271090430024 / (299)) + (1.5636347777316917 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.3619177857646961 = (185.3524701619441 - 118.27011457491639) / 185.3524701619441 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateFor(Int32).ImmutableSortedSet(Size%3a%20512).html>) 12.10 μs 10.49 μs 0.87 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.485135145168439 < 11.513646895976443. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 10.658460363075877 (T) = (0 -10556.5095380494) / Math.Sqrt((628072.0743363991 / (299)) + (51294.04398869448 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1009186719120044 = (11741.44007694953 - 10556.5095380494) / 11741.44007694953 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Compare_Same_Upper - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count%3a%201024%2c%20Options%3a%20(en-US%2c%20OrdinalIgnoreCase)).html>) 1.54 μs 1.19 μs 0.78 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1943775410769146 < 1.4614758399635555. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 24.856592760446656 (T) = (0 -1194.495029336023) / Math.Sqrt((18990.016844570117 / (299)) + (5.610111117802545 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1433196807738088 = (1394.3299531089588 - 1194.495029336023) / 1394.3299531089588 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/MicroBenchmarks.Serializers.XmlToStream(Location).XmlSerializer.html>) 1.94 μs 1.80 μs 0.93 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<Location>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.802556235061 < 1.8483927785887617. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/2/2022 7:01:35 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 22.908783303668013 (T) = (0 -1816.1063269686244) / Math.Sqrt((2352.8600297517323 / (299)) + (180.5690552658299 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07719885323391418 = (1968.0364868781162 - 1816.1063269686244) / 1968.0364868781162 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LastIndexOfAnyValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Char).LastIndexOfAnyValues(Size%3a%20512).html>) 97.75 ns 90.73 ns 0.93 0.11 False
[BinarySearch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Char).BinarySearch(Size%3a%20512).html>) 10.76 ns 9.08 ns 0.84 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 90.733449178304 < 92.31393217026724. IsChangePoint: Marked as a change because one of 3/15/2022 6:41:22 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 29.058750138498755 (T) = (0 -90.72285677340717) / Math.Sqrt((8.296176001014759 / (299)) + (0.02963183433865481 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.05551311514475201 = (96.05517898463074 - 90.72285677340717) / 96.05517898463074 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.BinarySearch(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.082187600061358 < 10.221155399990733. IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/7/2022 3:14:09 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 32.381984774407414 (T) = (0 -9.093340459117016) / Math.Sqrt((0.5054494371742451 / (299)) + (0.0009518507368095774 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1337780118979065 = (10.497702187219552 - 9.093340459117016) / 10.497702187219552 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Security.Cryptography.Tests.Perf_CryptoConfig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateFromName - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name%3a%20%22X509Chain%22).html>) 368.81 ns 286.11 ns 0.78 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Tests.Perf_CryptoConfig*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "X509Chain") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 286.112344935276 < 353.5866508443399. IsChangePoint: Marked as a change because one of 3/29/2022 11:07:04 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 69.05520710374387 (T) = (0 -288.4914075825862) / Math.Sqrt((160.875792725612 / (299)) + (4.685609180289248 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22524239388384137 = (372.3634402620282 - 288.4914075825862) / 372.3634402620282 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IndexerSetReverse(String).IList(Size%3a%20512).html>) 1.81 μs 1.45 μs 0.80 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSetReverse<String>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.4539040489620243 < 1.7279938241373909. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 14.784783776836168 (T) = (0 -1458.2265765820732) / Math.Sqrt((58447.960666309315 / (299)) + (100.17250034510324 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12955850224745868 = (1675.2723535667574 - 1458.2265765820732) / 1675.2723535667574 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsFalse(Int32).Stack(Size%3a%20512).html>) 80.57 μs 66.09 μs 0.82 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 66.09269646624473 < 76.4509614934796. IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 12.960205538618835 (T) = (0 -66139.45922819972) / Math.Sqrt((45597052.502823666 / (299)) + (1030.0794331855905 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07112685431104152 = (71203.97390662332 - 66139.45922819972) / 71203.97390662332 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in Benchstone.BenchF.InvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.BenchF.InvMt.Test.html>) 2.35 ms 1.92 ms 0.82 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.InvMt*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.InvMt.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.9181639014423078 < 2.235442169280563. IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 37.70004808899813 (T) = (0 -1918010.8005494506) / Math.Sqrt((27074259206.08944 / (299)) + (1442022.273584884 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15777928940806413 = (2277325.6183660217 - 1918010.8005494506) / 2277325.6183660217 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsTrue(Int32).Stack(Size%3a%20512).html>) 44.30 μs 37.10 μs 0.84 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 37.09549111374408 < 42.099833995720545. IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 19.872829261778865 (T) = (0 -37097.57262058834) / Math.Sqrt((12598451.029841725 / (299)) + (99.5757330233245 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09908822785395846 = (41177.80871285437 - 37097.57262058834) / 41177.80871285437 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Min - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Double.Min.html>) 6.50 μs 4.44 μs 0.68 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Double.Min ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 4.444623733743141 < 6.170071606830678. IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 29.429286303871585 (T) = (0 -4440.295266154679) / Math.Sqrt((801601.8326686359 / (299)) + (425.47145649732414 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.25847681653710614 = (5988.073421276751 - 4440.295266154679) / 5988.073421276751 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).Dictionary(Size%3a%20512).html>) 2.61 μs 2.30 μs 0.88 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyTrue<Int32, Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.3040225750049426 < 2.479236103202684. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 3/31/2022 1:23:38 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 18.135709685959203 (T) = (0 -2309.459221879589) / Math.Sqrt((59140.68235564801 / (299)) + (135.94135141526277 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10537559451630792 = (2581.484707686848 - 2309.459221879589) / 2581.484707686848 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in Benchstone.BenchI.TreeInsert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.BenchI.TreeInsert.Test.html>) 50.98 μs 47.09 μs 0.92 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.TreeInsert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.TreeInsert.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 47.08766795545546 < 48.471121165988606. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 4/20/2022 8:23:28 AM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 35.8631530831214 (T) = (0 -47006.8210384423) / Math.Sqrt((2200592.3490415225 / (299)) + (2164.4278085365027 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06309947951439668 = (50172.69177529998 - 47006.8210384423) / 50172.69177529998 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArraySet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Tests.Perf_BitArray.BitArraySet(Size%3a%20512).html>) 783.65 ns 613.89 ns 0.78 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArraySet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 613.8903984436565 < 744.1535036124568. IsChangePoint: Marked as a change because one of 3/2/2022 3:17:47 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 31.29058791277058 (T) = (0 -613.7390913271903) / Math.Sqrt((5176.029919723772 / (299)) + (0.016389067113910295 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.17501653734758157 = (743.9410838054223 - 613.7390913271903) / 743.9410838054223 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateInstance - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance.html>) 769.30 ns 644.82 ns 0.84 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 644.8237752231275 < 704.7063667528095. IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 4/8/2022 7:02:49 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 11.21558266357624 (T) = (0 -638.8007343803832) / Math.Sqrt((697.4769017572227 / (299)) + (523.7994325429994 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15375589975488285 = (754.8658054990902 - 638.8007343803832) / 754.8658054990902 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyFourValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Int32).IndexOfAnyFourValues(Size%3a%20512).html>) 622.53 ns 492.27 ns 0.79 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 492.2656714835348 < 591.5753930208626. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/7/2022 3:14:09 PM, 4/14/2022 8:50:18 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 22.56088570254251 (T) = (0 -498.2647611014281) / Math.Sqrt((3356.4086905194767 / (299)) + (11.8221773571734 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14304194012797405 = (581.4342433232224 - 498.2647611014281) / 581.4342433232224 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateForEach(String).ImmutableArray(Size%3a%20512).html>) 451.97 ns 228.84 ns 0.51 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 228.84123783493763 < 429.6143806561109. IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/29/2022 3:04:55 PM, 4/14/2022 8:50:18 PM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 22.946283253076583 (T) = (0 -228.9231057661994) / Math.Sqrt((11564.826596294542 / (299)) + (0.00660063255145075 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.3840074012310803 = (371.6328836153378 - 228.9231057661994) / 371.6328836153378 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff237352b2cc0996ab35d6110f7785e2329fe639
Compare 555f6112658ec729e9163f8c364868e8c353fafe
Diff Diff

Improvements in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Max - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.Max.html>) 5.44 μs 4.35 μs 0.80 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Single.Max ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 4.3544610866834175 < 5.172458881283498. IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/10/2022 8:56:34 AM falls between 5/1/2022 6:14:18 PM and 5/10/2022 8:56:34 AM. IsImprovementStdDev: Marked as improvement because 30.575163505254253 (T) = (0 -4357.848060048431) / Math.Sqrt((216670.74252300357 / (299)) + (10.339749009246912 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15905509445760913 = (5182.085094192604 - 4357.848060048431) / 5182.085094192604 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)