performanceautofiler[bot]
commented
1 year ago Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
| Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
| Diff | Diff |
Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(CollectionsOfPrimitives).SystemTextJsonSourceGen.html>) | 511.68 μs | 727.55 μs | 1.42 | 0.14 | False | |||||
| [SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(CollectionsOfPrimitives).SystemTextJsonReflection.html>) | 699.74 μs | 889.80 μs | 1.27 | 0.12 | False |
_1.png>)
_2.png>)
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 727.5472463768114 > 540.101609024738.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -62.48762882719696 (T) = (0 -723853.2283820749) / Math.Sqrt((27043204.912074927 / (14)) + (362102447.7732208 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -0.4007696668161248 = (516753.9285936673 - 723853.2283820749) / 516753.9285936673 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_Reflection_
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 889.7964669738865 > 746.8439654072505.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -39.696261709755746 (T) = (0 -937778.8843415609) / Math.Sqrt((131555772.52815673 / (14)) + (887168909.3583649 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -0.31210124617357804 = (714715.3370034236 - 937778.8843415609) / 714715.3370034236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
| Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
| Diff | Diff |
Regressions in System.Buffers.Text.Tests.Utf8FormatterTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%2012345).html>) | 17.21 ns | 121.30 ns | 7.05 | 0.60 | False | |||||
| [FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%209223372036854775807).html>) | 32.60 ns | 128.51 ns | 3.94 | 0.44 | False | |||||
| [FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%20-2147483648).html>) | 24.03 ns | 119.75 ns | 4.98 | 0.50 | False | |||||
| [FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2012345).html>) | 19.71 ns | 113.23 ns | 5.74 | 0.51 | False | |||||
| [FormatterUInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value%3a%204294967295).html>) | 22.62 ns | 125.07 ns | 5.53 | 0.50 | False | |||||
| [FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%2012345).html>) | 17.18 ns | 113.74 ns | 6.62 | 0.59 | False | |||||
| [FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2018446744073709551615).html>) | 30.76 ns | 122.64 ns | 3.99 | 0.38 | False | |||||
| [FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%20-9223372036854775808).html>) | 29.87 ns | 127.57 ns | 4.27 | 0.46 | False | |||||
| [FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%202147483647).html>) | 23.38 ns | 125.04 ns | 5.35 | 0.49 | False | |||||
| [FormatterUInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value%3a%2012345).html>) | 19.25 ns | 114.54 ns | 5.95 | 0.51 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.3019725811671 > 18.234178105141815.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -103.96766385486474 (T) = (0 -114.51879255848715) / Math.Sqrt((1.9427309567800217 / (14)) + (28.80255891530659 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -5.303071361838121 = (18.168728542697448 - 114.51879255848715) / 18.168728542697448 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 128.50735353076178 > 31.49322717048142.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -114.81564676312838 (T) = (0 -126.74865899482249) / Math.Sqrt((2.004019328953629 / (14)) + (22.40244972806198 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -3.1601536191863615 = (30.467302555911832 - 126.74865899482249) / 30.467302555911832 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: -2147483648)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 119.75278980924408 > 25.183537366728274.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -83.96268520867363 (T) = (0 -121.84945406663546) / Math.Sqrt((2.989469022859104 / (14)) + (45.42075704961862 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -4.008635715352298 = (24.327873095890503 - 121.84945406663546) / 24.327873095890503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.2300734095382 > 19.228359248163617.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -100.68283531428176 (T) = (0 -112.62032838570735) / Math.Sqrt((1.5571387695481522 / (14)) + (31.034566438575123 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -5.329890372239483 = (17.791829204438947 - 112.62032838570735) / 17.791829204438947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 125.06550510954273 > 23.810160173953978.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -90.61402127500958 (T) = (0 -120.91920345803018) / Math.Sqrt((2.7552414439585613 / (14)) + (38.71768396746822 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -4.228822888126684 = (23.125511428701607 - 120.91920345803018) / 23.125511428701607 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.7357823724508 > 20.15686288133497.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -100.75052858261982 (T) = (0 -114.02281712705687) / Math.Sqrt((2.4466884280002548 / (14)) + (28.77558139982546 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -5.0800960905008035 = (18.753456430598135 - 114.02281712705687) / 18.753456430598135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 122.64078121654477 > 32.06694366085135.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -92.10863783968834 (T) = (0 -128.6995942127358) / Math.Sqrt((1.999921322673542 / (14)) + (39.34822769188945 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -3.1601833446268426 = (30.93603900389631 - 128.6995942127358) / 30.93603900389631 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.57176721792258 > 31.808571164824784.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -89.36394053072247 (T) = (0 -125.94408843603314) / Math.Sqrt((5.082085438889313 / (14)) + (29.868508065132154 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -2.95985077900515 = (31.80526122443296 - 125.94408843603314) / 31.80526122443296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 125.04156490070238 > 24.437023928579748.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -90.47683543971235 (T) = (0 -118.53101195843342) / Math.Sqrt((2.4881631172589804 / (14)) + (36.55082012691061 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -3.9376393832010903 = (24.005603236579283 - 118.53101195843342) / 24.005603236579283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 12345)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.54294501062074 > 18.39916633763085.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -90.27405200378641 (T) = (0 -111.99202751845303) / Math.Sqrt((1.0605868323798653 / (14)) + (40.47044899244568 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -5.273872816619266 = (17.85054157007935 - 111.99202751845303) / 17.85054157007935 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
| Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
| Diff | Diff |
Regressions in System.Tests.Perf_SByte
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_SByte.ToString(value%3a%20127).html>) | 63.37 ns | 162.82 ns | 2.57 | 0.34 | False | |||||
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_SByte.ToString(value%3a%20-128).html>) | 77.63 ns | 178.35 ns | 2.30 | 0.35 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Tests.Perf_SByte.ToString(value: 127)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 162.8238706173419 > 71.46309095702445.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -80.90493233902185 (T) = (0 -161.15677269489447) / Math.Sqrt((7.242047874720535 / (14)) + (33.56735803240088 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -1.4078895835667844 = (66.92863900186588 - 161.15677269489447) / 66.92863900186588 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_SByte.ToString(value: -128)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 178.35179638321765 > 80.02424123002353.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -86.81006966828728 (T) = (0 -174.15731601884013) / Math.Sqrt((9.522076582939153 / (14)) + (22.14117001315438 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -1.2403553617091658 = (77.73646939920086 - 174.15731601884013) / 77.73646939920086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 2eca2d319bf8d1247edf38facfdd501e0e2f0030 |
| Compare | 51a8dd5323721b363e61069575511f783e7ea6d3 |
| Diff | Diff |
Regressions in System.Tests.Perf_Int32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryFormat(value%3a%20-2147483648).html>) | 48.01 ns | 146.64 ns | 3.05 | 0.31 | False | |||||
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ToString(value%3a%202147483647).html>) | 70.69 ns | 162.48 ns | 2.30 | 0.27 | False | |||||
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ToString(value%3a%20-2147483648).html>) | 80.12 ns | 178.37 ns | 2.23 | 0.24 | False | |||||
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryFormat(value%3a%2012345).html>) | 26.95 ns | 119.04 ns | 4.42 | 0.60 | False | |||||
| [ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ToString(value%3a%2012345).html>) | 65.35 ns | 164.84 ns | 2.52 | 0.39 | False | |||||
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryFormat(value%3a%202147483647).html>) | 24.89 ns | 126.96 ns | 5.10 | 0.48 | False | |||||
| [TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryFormat(value%3a%204).html>) | 20.24 ns | 127.25 ns | 6.29 | 0.59 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Tests.Perf_Int32.TryFormat(value: -2147483648)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 146.63748334119288 > 50.28344603027571.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -92.13710310461555 (T) = (0 -142.72098238761708) / Math.Sqrt((5.573286016126088 / (14)) + (27.841643021154933 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -2.079545063531046 = (46.34482673358622 - 142.72098238761708) / 46.34482673358622 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToString(value: 2147483647)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 162.4751087838336 > 75.05893636231025.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -96.23941424343826 (T) = (0 -166.76175938084677) / Math.Sqrt((6.797851463078619 / (14)) + (20.403621819207277 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -1.3576622238373555 = (70.73182820456087 - 166.76175938084677) / 70.73182820456087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToString(value: -2147483648)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 178.37305827853237 > 84.52968027527372.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -72.70005002097079 (T) = (0 -181.4992699587163) / Math.Sqrt((5.084621584525473 / (14)) + (61.78504388909607 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -1.2384288705222541 = (81.08333141556122 - 181.4992699587163) / 81.08333141556122 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryFormat(value: 12345)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 119.03754658224481 > 26.247572633115833.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -107.7662841190693 (T) = (0 -121.7155161200729) / Math.Sqrt((6.037137101863773 / (14)) + (14.269264006752293 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -3.671526808444585 = (26.054761346986442 - 121.7155161200729) / 26.054761346986442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToString(value: 12345)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 164.83872653555164 > 69.95331848206757.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -94.10373182516433 (T) = (0 -162.81559554612753) / Math.Sqrt((2.81432788041265 / (14)) + (34.15443175384841 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -1.460790057543501 = (66.16395212058814 - 162.81559554612753) / 66.16395212058814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryFormat(value: 2147483647)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.95770035757347 > 26.520179229591168.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -99.7590522171166 (T) = (0 -125.5356125790175) / Math.Sqrt((3.7530618933906794 / (14)) + (28.798873264329078 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -3.7597291655661627 = (26.374528510402172 - 125.5356125790175) / 26.374528510402172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryFormat(value: 4)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.2455632103405 > 23.112881717699686.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/24/2023 6:33:57 AM falls between 1/15/2023 4:08:29 PM and 1/24/2023 6:33:57 AM.
IsRegressionStdDev: Marked as regression because -93.71766491375244 (T) = (0 -119.0638848487287) / Math.Sqrt((6.062398638964241 / (14)) + (25.53219579005241 / (40))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (40) - 2, .025) and -4.396982454384553 = (22.06119546525489 - 119.0638848487287) / 22.06119546525489 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
_1.png>)
_2.png>)
_3.png>)
_4.png>)
_5.png>)
_6.png>)
_7.png>)
_8.png>)
_1.png>)
_2.png>)
_1.png>)
radekdoulik
matouskozak
Run Information
Regressions in System.Tests.Perf_Version
Repro
Run Information
Regressions in System.Text.Json.Tests.Perf_Deep
Repro