performanceautofiler[bot]
commented
2 years ago Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | cb3bd225a88dd75ad804a3701a2ac9b2553a4973 |
| Compare | f9e9d86e77210bee1779970273d124ceeae0df7e |
| Diff | Diff |
Improvements in System.Collections.Concurrent.Count<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Concurrent.Count(String).Queue(Size%3a%20512).html>) | 81.16 ns | 55.28 ns | 0.68 | 0.17 | False | |||||
| [Bag - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Concurrent.Count(String).Bag(Size%3a%20512).html>) | 517.26 ns | 446.67 ns | 0.86 | 0.11 | False | |||||
| [Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Concurrent.Count(String).Stack(Size%3a%20512).html>) | 2.69 μs | 1.76 μs | 0.65 | 0.07 | False | |||||
| [Queue_EnqueueCountDequeue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.Concurrent.Count(String).Queue_EnqueueCountDequeue(Size%3a%20512).html>) | 317.15 ns | 246.18 ns | 0.78 | 0.13 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.Count<String>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Collections.Concurrent.Count<String>.Queue(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 55.28028216846345 < 77.89688128901591.
IsChangePoint: Marked as a change because one of 5/25/2022 7:30:00 PM, 6/5/2022 6:02:14 PM, 6/7/2022 3:01:09 PM falls between 5/29/2022 5:15:01 PM and 6/7/2022 3:01:09 PM.
IsImprovementStdDev: Marked as improvement because 8.002269575943599 (T) = (0 -57.091207992587435) / Math.Sqrt((102.56199968380966 / (299)) + (4.619728387884812 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.10681874500105747 = (63.918950014972076 - 57.091207992587435) / 63.918950014972076 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Concurrent.Count<String>.Bag(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 446.67092603794737 < 485.3125990059153.
IsChangePoint: Marked as a change because one of 5/25/2022 7:30:00 PM, 6/5/2022 6:02:14 PM, 6/7/2022 3:01:09 PM falls between 5/29/2022 5:15:01 PM and 6/7/2022 3:01:09 PM.
IsImprovementStdDev: Marked as improvement because 9.849964822142283 (T) = (0 -452.9700820235671) / Math.Sqrt((390.2072121484355 / (299)) + (100.46874309769554 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.06336038936351543 = (483.6119216821885 - 452.9700820235671) / 483.6119216821885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Concurrent.Count<String>.Stack(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7560876322939343 < 2.572911568060244.
IsChangePoint: Marked as a change because one of 5/25/2022 7:30:00 PM, 6/5/2022 6:02:14 PM, 6/7/2022 3:01:09 PM falls between 5/29/2022 5:15:01 PM and 6/7/2022 3:01:09 PM.
IsImprovementStdDev: Marked as improvement because 11.324182013032127 (T) = (0 -1740.7991310240911) / Math.Sqrt((141102.98531433073 / (299)) + (4533.792769893057 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.15940010438300636 = (2070.900960255721 - 1740.7991310240911) / 2070.900960255721 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Concurrent.Count<String>.Queue_EnqueueCountDequeue(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 246.17644007648252 < 300.92564672404694.
IsChangePoint: Marked as a change because one of 5/25/2022 7:30:00 PM, 6/5/2022 6:02:14 PM, 6/7/2022 3:01:09 PM falls between 5/29/2022 5:15:01 PM and 6/7/2022 3:01:09 PM.
IsImprovementStdDev: Marked as improvement because 15.543637138552763 (T) = (0 -246.73783729492638) / Math.Sqrt((782.9118713164776 / (299)) + (12.433145060787494 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.10748506864019068 = (276.45233555813337 - 246.73783729492638) / 276.45233555813337 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>
Test Report
Repro