performanceautofiler[bot]
commented
1 year ago Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 0579209324586b993ca417435cf70fd44213cadc |
| Compare | e467a5f65a4fb6b0b703a5c1c22c519114e99845 |
| Diff | Diff |
Regressions in System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) | 188.86 μs | 226.63 μs | 1.20 | 0.03 | False | |||||
| [DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromString(Mode%3a%20Reflection).html>) | 225.37 μs | 254.77 μs | 1.13 | 0.03 | False | |||||
| [DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromStream(Mode%3a%20SourceGen).html>) | 212.63 μs | 244.27 μs | 1.15 | 0.03 | False | |||||
| [DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) | 189.21 μs | 229.47 μs | 1.21 | 0.07 | False | |||||
| [DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromString(Mode%3a%20SourceGen).html>) | 226.15 μs | 253.42 μs | 1.12 | 0.03 | False | |||||
| [DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromStream(Mode%3a%20Reflection).html>) | 210.89 μs | 245.02 μs | 1.16 | 0.03 | False | |||||
| [DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromReader(Mode%3a%20SourceGen).html>) | 266.50 μs | 297.24 μs | 1.12 | 0.10 | False | |||||
| [DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromReader(Mode%3a%20Reflection).html>) | 263.92 μs | 300.51 μs | 1.14 | 0.03 | False |
Test Report.html>)
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromUtf8Bytes(Mode: SourceGen)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -19.75001436862547 (T) = (0 -227157.71196895427) / Math.Sqrt((21505900.85662481 / (6)) + (402217.47316776344 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.20142536280951065 = (189073.5113480127 - 227157.71196895427) / 189073.5113480127 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString(Mode: Reflection)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -10.977032361868874 (T) = (0 -255659.30078703703) / Math.Sqrt((43458393.579892725 / (6)) + (3468843.06703968 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.14212098577582324 = (223846.07582827317 - 255659.30078703703) / 223846.07582827317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromStream(Mode: SourceGen)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -13.61704727921951 (T) = (0 -244280.0126443938) / Math.Sqrt((36812697.24742985 / (6)) + (8172.547814243753 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.1602363600530345 = (210543.3177712408 - 244280.0126443938) / 210543.3177712408 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromUtf8Bytes(Mode: Reflection)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -12.08647444316135 (T) = (0 -227323.66177181317) / Math.Sqrt((49987176.456827104 / (6)) + (3457819.466092312 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.19579574565839303 = (190102.4172373611 - 227323.66177181317) / 190102.4172373611 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString(Mode: SourceGen)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -9.60128993551371 (T) = (0 -254762.19833348025) / Math.Sqrt((58238126.015291944 / (6)) + (2097219.3516052584 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.13839396219589423 = (223790.89031890078 - 254762.19833348025) / 223790.89031890078 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromStream(Mode: Reflection)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -15.797002704754467 (T) = (0 -244989.4588216146) / Math.Sqrt((28879568.1474407 / (6)) + (64189.76796968778 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.165200267569167 = (210255.2373530689 - 244989.4588216146) / 210255.2373530689 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromReader(Mode: SourceGen)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -11.199471182564238 (T) = (0 -297641.5552170652) / Math.Sqrt((56357552.00666231 / (6)) + (181552.6625722761 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.13082623030997048 = (263207.1552987224 - 297641.5552170652) / 263207.1552987224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromReader(Mode: Reflection)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -13.345834629964768 (T) = (0 -298758.88713369967) / Math.Sqrt((37440198.252901316 / (6)) + (2335897.0400559837 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.13423130236288874 = (263402.08254816267 - 298758.88713369967) / 263402.08254816267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 0579209324586b993ca417435cf70fd44213cadc |
| Compare | e467a5f65a4fb6b0b703a5c1c22c519114e99845 |
| Diff | Diff |
Regressions in MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.JsonFromStream(IndexViewModel).JsonNet.html>) | 902.12 μs | 1.02 ms | 1.13 | 0.02 | False | |||||
| [SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.Json_FromStream(IndexViewModel).SystemTextJsonSourceGen.html>) | 577.68 μs | 646.62 μs | 1.12 | 0.03 | False | |||||
| [SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/MicroBenchmarks.Serializers.Json_FromStream(IndexViewModel).SystemTextJsonReflection.html>) | 573.66 μs | 636.12 μs | 1.11 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.JsonNet_
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -38.247149285420804 (T) = (0 -1018563.7536907537) / Math.Sqrt((42188409.64675452 / (6)) + (4921475.389236255 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.12432539221865017 = (905933.247385621 - 1018563.7536907537) / 905933.247385621 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_SourceGen_
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -8.022959269399996 (T) = (0 -645554.9224828626) / Math.Sqrt((524539678.6380893 / (6)) + (2546765.1839434034 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.13220174554361322 = (570176.5829489223 - 645554.9224828626) / 570176.5829489223 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_Reflection_
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -10.419245917003412 (T) = (0 -638289.5213031421) / Math.Sqrt((295230398.3097536 / (6)) + (13852890.743219428 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.13605007123012788 = (561849.8140772923 - 638289.5213031421) / 561849.8140772923 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 0579209324586b993ca417435cf70fd44213cadc |
| Compare | e467a5f65a4fb6b0b703a5c1c22c519114e99845 |
| Diff | Diff |
Regressions in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [CachedJsonSerializerContext - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).CachedJsonSerializerContext.html>) | 10.75 μs | 11.36 μs | 1.06 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedJsonSerializerContext
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -13.61291800493817 (T) = (0 -11353.263960820777) / Math.Sqrt((5973.192132813552 / (6)) + (3545.255106591258 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.05926418190955836 = (10718.066517036386 - 11353.263960820777) / 10718.066517036386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 0579209324586b993ca417435cf70fd44213cadc |
| Compare | e467a5f65a4fb6b0b703a5c1c22c519114e99845 |
| Diff | Diff |
Regressions in System.IO.Tests.StreamReaderReadLineTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>) | 197.16 μs | 364.62 μs | 1.85 | 0.24 | False | |||||
| [ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>) | 200.12 μs | 358.17 μs | 1.79 | 0.14 | False | |||||
| [ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>) | 233.99 μs | 394.16 μs | 1.68 | 0.20 | False | |||||
| [ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>) | 412.07 μs | 571.02 μs | 1.39 | 0.06 | False | |||||
| [ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>) | 238.23 μs | 396.47 μs | 1.66 | 0.18 | False | |||||
| [ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>) | 236.69 μs | 392.99 μs | 1.66 | 0.14 | False | |||||
| [ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%209%2c%20%20%2032%5d).html>) | 493.41 μs | 656.02 μs | 1.33 | 0.08 | False | |||||
| [ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>) | 204.64 μs | 368.70 μs | 1.80 | 0.20 | False | |||||
| [ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%20%209%2c%20%20%2032%5d).html>) | 977.46 μs | 1.14 ms | 1.17 | 0.06 | False | |||||
| [ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>) | 271.61 μs | 411.76 μs | 1.52 | 0.14 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 11/6/2022 11:08:52 PM, 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -43.43039689772255 (T) = (0 -360286.6078564683) / Math.Sqrt((21327287.750175834 / (6)) + (29859662.27091826 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.7954431468000129 = (200667.23276568286 - 360286.6078564683) / 200667.23276568286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 11/6/2022 11:08:52 PM, 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -60.03607071926773 (T) = (0 -356087.57923836936) / Math.Sqrt((32014313.427123122 / (6)) + (4166830.2405866366 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.7768635790753506 = (200402.3175620895 - 356087.57923836936) / 200402.3175620895 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -28.160263965702747 (T) = (0 -392082.58571908885) / Math.Sqrt((68719917.08411689 / (6)) + (64399140.08906105 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.7009269581567072 = (230511.12444239718 - 392082.58571908885) / 230511.12444239718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 33, 128])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -80.46123132580216 (T) = (0 -569447.2637444168) / Math.Sqrt((11219736.368127242 / (6)) + (5212928.689264794 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.3668207028780573 = (416621.77236952545 - 569447.2637444168) / 416621.77236952545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -87.71526990235647 (T) = (0 -395726.52965370286) / Math.Sqrt((13131587.843954075 / (6)) + (3106947.501418051 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.6611577068086008 = (238223.33546762916 - 395726.52965370286) / 238223.33546762916 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 1024])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -93.50143171767657 (T) = (0 -392787.288345067) / Math.Sqrt((15304547.374174273 / (6)) + (320636.3173296752 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.6341704299359074 = (240358.82742075576 - 392787.288345067) / 240358.82742075576 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -29.573527434961743 (T) = (0 -653119.9375032041) / Math.Sqrt((141269306.37418354 / (6)) + (6294626.1848035585 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.2975140335072349 = (503362.522976182 - 653119.9375032041) / 503362.522976182 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 11/6/2022 11:08:52 PM, 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -36.70789633089185 (T) = (0 -367581.2638491327) / Math.Sqrt((19668260.77561231 / (6)) + (49574274.93693847 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.7998407623823257 = (204229.88051597998 - 367581.2638491327) / 204229.88051597998 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 9, 32])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -19.538542959678214 (T) = (0 -1143551.792966361) / Math.Sqrt((328550409.00427717 / (6)) + (12726218.364447128 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.15106503204647453 = (993472.7935686172 - 1143551.792966361) / 993472.7935686172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128])
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -44.583249859021464 (T) = (0 -413215.81500735955) / Math.Sqrt((54576837.50563139 / (6)) + (4257361.497915794 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.5381502718006423 = (268644.6328313726 - 413215.81500735955) / 268644.6328313726 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 0579209324586b993ca417435cf70fd44213cadc |
| Compare | e467a5f65a4fb6b0b703a5c1c22c519114e99845 |
| Diff | Diff |
Regressions in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToString(Mode%3a%20SourceGen).html>) | 3.85 μs | 4.25 μs | 1.10 | 0.07 | False | |||||
| [SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeObjectProperty(Mode%3a%20Reflection).html>) | 8.29 μs | 8.77 μs | 1.06 | 0.03 | False | |||||
| [SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeObjectProperty(Mode%3a%20SourceGen).html>) | 7.55 μs | 7.96 μs | 1.05 | 0.03 | False | |||||
| [SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToString(Mode%3a%20Reflection).html>) | 5.16 μs | 5.56 μs | 1.08 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString(Mode: SourceGen)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -10.387806185881814 (T) = (0 -4292.967828954627) / Math.Sqrt((982.7765181981767 / (6)) + (4860.698386828686 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.11383855992414085 = (3854.2101013696315 - 4292.967828954627) / 3854.2101013696315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: Reflection)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -11.592221688341866 (T) = (0 -8844.4750352852) / Math.Sqrt((1427.0139409220278 / (6)) + (5787.392167821046 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.06497720342365995 = (8304.849161890246 - 8844.4750352852) / 8304.849161890246 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: SourceGen)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -17.5294221688404 (T) = (0 -7987.351980973471) / Math.Sqrt((3811.2046500721613 / (6)) + (734.8926730715345 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.06964440776017275 = (7467.296536144124 - 7987.351980973471) / 7467.296536144124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString(Mode: Reflection)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/20/2022 11:23:14 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -36.24282918073683 (T) = (0 -5556.818793999803) / Math.Sqrt((531.7778193590244 / (6)) + (98.96019282198216 / (3))) is less than -2.3646242515927898 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (3) - 2, .025) and -0.0775014320057188 = (5157.133558194947 - 5556.818793999803) / 5157.133558194947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kotlarmilos
Run Information
Regressions in System.IO.Tests.StreamReaderReadToEndTests
Test Report
Repro
Run Information
Regressions in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks
Test Report
Repro
Run Information
Regressions in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>
Test Report
Repro
Run Information
Regressions in System.Text.Json.Serialization.Tests.WriteJson<Location>
Test Report
Repro