[Perf] Linux/x64: 8 Improvements on 7/10/2023 5:20:40 AM

[performanceautofiler[bot]]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	bd634023d2c09950721e29f75b70b4f3a9819d77
Compare	36481bfd079093fe7d29ef024b1fb801e6c787fe
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(LoginViewModel).SystemTextJsonReflection.html>)	7.54 μs	0.00 ns	0.00	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>*'

### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 7.115070976099057. IsChangePoint: Marked as a change because one of 4/5/2023 6:15:03 PM, 4/6/2023 9:09:21 PM, 7/6/2023 7:42:33 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM. IsImprovementStdDev: Marked as improvement because 107.24264791817266 (T) = (0 -0) / Math.Sqrt((1591993.2891313054 / (299)) + (0 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 1 = (7825.328011148612 - 0) / 7825.328011148612 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	bd634023d2c09950721e29f75b70b4f3a9819d77
Compare	36481bfd079093fe7d29ef024b1fb801e6c787fe
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(CollectionsOfPrimitives).SystemTextJsonReflection.html>)	2.46 ms	0.00 ns	0.00	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>*'

### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_Reflection_ #### ETL Files #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 2.3410545023419043. IsChangePoint: Marked as a change because one of 7/6/2023 7:42:33 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM. IsImprovementStdDev: Marked as improvement because 50.353904456138125 (T) = (0 -0) / Math.Sqrt((835072571075.6815 / (299)) + (0 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 1 = (2661090.6366559262 - 0) / 2661090.6366559262 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	bd634023d2c09950721e29f75b70b4f3a9819d77
Compare	36481bfd079093fe7d29ef024b1fb801e6c787fe
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(MyEventsListerViewModel).SystemTextJsonReflection.html>)	8.09 ms	0.00 ns	0.00	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>*'

### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 7.693877746436781. IsChangePoint: Marked as a change because one of 3/22/2023 7:51:01 PM, 3/23/2023 10:59:44 AM, 7/6/2023 7:42:33 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM. IsImprovementStdDev: Marked as improvement because 89.24291668956971 (T) = (0 -0) / Math.Sqrt((2600163552678.1357 / (299)) + (0 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 1 = (8322206.632781807 - 0) / 8322206.632781807 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	bd634023d2c09950721e29f75b70b4f3a9819d77
Compare	36481bfd079093fe7d29ef024b1fb801e6c787fe
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(CollectionsOfPrimitives).SystemTextJsonReflection.html>)	2.42 ms	0.00 ns	0.00	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>*'

### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ #### ETL Files #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 2.299028430657096. IsChangePoint: Marked as a change because one of 4/11/2023 10:44:16 AM, 4/12/2023 9:47:42 AM, 7/6/2023 7:42:33 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM. IsImprovementStdDev: Marked as improvement because 69.69493936142021 (T) = (0 -0) / Math.Sqrt((387417428246.16174 / (299)) + (0 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 1 = (2508737.5331871687 - 0) / 2508737.5331871687 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	bd634023d2c09950721e29f75b70b4f3a9819d77
Compare	36481bfd079093fe7d29ef024b1fb801e6c787fe
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(LoginViewModel).SystemTextJsonReflection.html>)	6.86 μs	0.00 ns	0.00	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>*'

### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 6.579189909785921. IsChangePoint: Marked as a change because one of 7/6/2023 7:42:33 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM. IsImprovementStdDev: Marked as improvement because 95.86399426626228 (T) = (0 -0) / Math.Sqrt((1644711.243179039 / (299)) + (0 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 1 = (7109.9206985766505 - 0) / 7109.9206985766505 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	bd634023d2c09950721e29f75b70b4f3a9819d77
Compare	36481bfd079093fe7d29ef024b1fb801e6c787fe
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<Location>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(Location).SystemTextJsonReflection.html>)	15.35 μs	0.00 ns	0.00	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<Location>*'

### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 14.808340681634641. IsChangePoint: Marked as a change because one of 3/26/2023 7:47:31 AM, 3/27/2023 4:20:55 AM, 4/9/2023 11:04:55 PM, 4/10/2023 8:15:35 PM, 7/6/2023 7:42:33 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM. IsImprovementStdDev: Marked as improvement because 108.80712816075844 (T) = (0 -0) / Math.Sqrt((6407220.336257403 / (299)) + (0 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 1 = (15927.828950092287 - 0) / 15927.828950092287 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	bd634023d2c09950721e29f75b70b4f3a9819d77
Compare	36481bfd079093fe7d29ef024b1fb801e6c787fe
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(MyEventsListerViewModel).SystemTextJsonReflection.html>)	8.09 ms	0.00 ns	0.00	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>*'

### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 7.764201327841308. IsChangePoint: Marked as a change because one of 7/6/2023 7:42:33 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM. IsImprovementStdDev: Marked as improvement because 63.69185099832795 (T) = (0 -0) / Math.Sqrt((5620169230742.713 / (299)) + (0 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 1 = (8732185.897883803 - 0) / 8732185.897883803 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	bd634023d2c09950721e29f75b70b4f3a9819d77
Compare	36481bfd079093fe7d29ef024b1fb801e6c787fe
Diff	Diff
Configs	CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<Location>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(Location).SystemTextJsonReflection.html>)	16.42 μs	0.00 ns	0.00	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString<Location>*'

### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 15.781402952385928. IsChangePoint: Marked as a change because one of 3/24/2023 7:28:10 PM, 3/25/2023 6:47:18 AM, 7/6/2023 7:42:33 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM. IsImprovementStdDev: Marked as improvement because 66.59778697297529 (T) = (0 -0) / Math.Sqrt((20374846.775815636 / (299)) + (0 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 1 = (17384.867277361926 - 0) / 17384.867277361926 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

[radekdoulik]

@radical @LoopedBard3 these results look weird. the times are now zero. is it some infrastructure issue?

just for the record, the runtime range is https://github.com/dotnet/runtime/compare/0224f86a886e0aaf72acb6a4f5573236ba929292...36481bfd079093fe7d29ef024b1fb801e6c787fe

[LoopedBard3]

@cincuranet Do you know if we have seen similar improvements anywhere else with the root cause determined?

[cincuranet]

@LoopedBard3 At the moment I can speak only about x64 and I haven't seen this change there.

[cincuranet]

Maybe it's related to https://github.com/dotnet/performance/pull/3115...

[LoopedBard3]

I found a log with this failure under the run here: https://dev.azure.com/dnceng/internal/_build/results?buildId=2218001&view=results (20230710.4). An error was hit causing the benchmark to fail its run:

[2023/07/10 13:41:04][INFO] // Job: Job-QXFDDW(PowerPlanMode=00000000-0000-0000-0000-000000000000, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)
[2023/07/10 13:41:04][INFO] System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
[2023/07/10 13:41:04][INFO]  ---> System.NotSupportedException: JsonTypeInfo metadata for type 'MicroBenchmarks.Serializers.CollectionsOfPrimitives' was not provided by TypeInfoResolver of type '[]'. If using source generation, ensure that all root types passed to the serializer have been annotated with 'JsonSerializableAttribute', along with any types that might be serialized polymorphically.
[2023/07/10 13:41:04][INFO]    at System.Text.Json.ThrowHelper.ThrowNotSupportedException_NoMetadataForType(Type , IJsonTypeInfoResolver )
[2023/07/10 13:41:04][INFO]    at System.Text.Json.JsonSerializerOptions.GetTypeInfoInternal(Type , Boolean , Nullable`1 , Boolean , Boolean )
[2023/07/10 13:41:04][INFO]    at System.Text.Json.JsonSerializerOptions.GetTypeInfoForRootType(Type , Boolean )
[2023/07/10 13:41:04][INFO]    at System.Text.Json.JsonSerializer.GetTypeInfo(JsonSerializerOptions , Type )
[2023/07/10 13:41:04][INFO]    at System.Text.Json.JsonSerializer.GetTypeInfo[CollectionsOfPrimitives](JsonSerializerOptions )
[2023/07/10 13:41:04][INFO]    at System.Text.Json.JsonSerializer.Serialize[CollectionsOfPrimitives](Stream , CollectionsOfPrimitives , JsonSerializerOptions )
[2023/07/10 13:41:04][INFO]    at MicroBenchmarks.Serializers.Json_FromStream`1[[MicroBenchmarks.Serializers.CollectionsOfPrimitives, MicroBenchmarks, Version=42.42.42.42, Culture=neutral, PublicKeyToken=null]].SetupSystemTextJson_Reflection_()
[2023/07/10 13:41:04][INFO]    at BenchmarkDotNet.Engines.EngineFactory.CreateReadyToRun(EngineParameters engineParameters)
[2023/07/10 13:41:04][INFO]    at BenchmarkDotNet.Autogenerated.Runnable_8.Run(IHost host, String benchmarkName)
[2023/07/10 13:41:04][INFO]    at System.Reflection.MethodInvoker.InterpretedInvoke(Object , IntPtr* )
[2023/07/10 13:41:04][INFO]    at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
[2023/07/10 13:41:04][INFO]    --- End of inner exception stack trace ---
[2023/07/10 13:41:04][INFO]    at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
[2023/07/10 13:41:04][INFO]    at System.Reflection.RuntimeMethodInfo.Invoke(Object , BindingFlags , Binder , Object[] , CultureInfo )
[2023/07/10 13:41:04][INFO]    at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
[2023/07/10 13:41:04][INFO]    at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
[2023/07/10 13:41:04][INFO] // AfterAll
[2023/07/10 13:41:04][INFO] test-main.js exiting 9be6d396-2873-49f0-afdc-0f119e16e9f1.dll --benchmarkName with result -1
[2023/07/10 13:41:04][INFO] MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
[2023/07/10 13:41:04][INFO] console.info: WASM EXIT -1
[2023/07/10 13:41:04][INFO] No Workload Results were obtained from the run.

The latest runs are not hitting this error. https://github.com/dotnet/performance/pull/3115 appears to have fixed the issue so we should see a regression on these same values next autofile run. I have opened an issue to check out the NA values being inserted as 0.

[radekdoulik]

browser-bench was affected by this too, should be fixed with https://github.com/dotnet/runtime/commit/de1a93a4765d852d30a7fa9f91216607cdfa8a6b