search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 24 Improvements on 3/16/2023 1:49:47 PM #14335

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7f0c19acfcfe1d89d1aaee814e6fd00b3ff1c8a2
Compare ef43a7b596545602a8bfe47d7a2b27a8ab5516ae
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(CollectionsOfPrimitives).SystemTextJsonReflection.html>) 2.81 ms 2.33 ms 0.83 0.06 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToStream(CollectionsOfPrimitives).JsonNet.html>) 7.05 ms 5.78 ms 0.82 0.04 False
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToStream(CollectionsOfPrimitives).DataContractJsonSerializer.html>) 50.62 ms 38.28 ms 0.76 0.08 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.3290822082210245 < 2.71807884734058. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/15/2023 2:53:22 PM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 3.6712930269967705 (T) = (0 -2713069.9964661933) / Math.Sqrt((1410100747088.6687 / (299)) + (2128841585623.0835 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3059179660814814 = (3908860.716577333 - 2713069.9964661933) / 3908860.716577333 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 5.78250680782313 < 6.7783298299369745. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/16/2023 2:58:09 AM, 3/16/2023 1:49:47 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 3.7261829221489196 (T) = (0 -6436702.382441949) / Math.Sqrt((1942354387768.5344 / (299)) + (7966506159424.961 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.26918006777619036 = (8807507.97649392 - 6436702.382441949) / 8807507.97649392 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 38.28146157692308 < 47.863366492410705. IsChangePoint: Marked as a change because one of 2/15/2023 8:55:14 PM, 3/17/2023 8:17:18 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 3.4741657599010143 (T) = (0 -42302465.92084478) / Math.Sqrt((22476105029042.203 / (299)) + (245279083065144.12 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.24376615809184152 = (55938340.20189517 - 42302465.92084478) / 55938340.20189517 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7f0c19acfcfe1d89d1aaee814e6fd00b3ff1c8a2
Compare ef43a7b596545602a8bfe47d7a2b27a8ab5516ae
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Ctor4_ActivatorCreateInstance - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Invoke.Ctor4_ActivatorCreateInstance.html>) 11.19 μs 9.82 μs 0.88 0.07 False
[Field_Get_int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Invoke.Field_Get_int.html>) 2.14 μs 1.93 μs 0.90 0.07 False
[Method0_NoParms - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Invoke.Method0_NoParms.html>) 1.16 μs 1.01 μs 0.87 0.15 False
[Property_Get_int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Invoke.Property_Get_int.html>) 151.51 ns 127.25 ns 0.84 0.10 False
[Ctor0_NoParams - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Invoke.Ctor0_NoParams.html>) 499.90 ns 441.15 ns 0.88 0.15 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Reflection.Invoke.Ctor4_ActivatorCreateInstance ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.817293961538462 < 10.713524449388526. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/15/2023 4:14:49 PM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 4.501552038947297 (T) = (0 -10342.888649416876) / Math.Sqrt((2360570.776882202 / (299)) + (8422064.330424974 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.21772669846226197 = (13221.579503078465 - 10342.888649416876) / 13221.579503078465 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Invoke.Field_Get_int ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.9278906276041667 < 1.9893920678999255. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/15/2023 2:53:22 PM, 3/16/2023 2:58:09 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 18.377380623941015 (T) = (0 -1968.4642726645177) / Math.Sqrt((120619.33744758995 / (299)) + (3089.3273362724985 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1779626427961694 = (2394.616565067397 - 1968.4642726645177) / 2394.616565067397 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Invoke.Method0_NoParms ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.01176484453125 < 1.1495213136458333. IsChangePoint: Marked as a change because one of 2/1/2023 10:12:25 AM, 3/10/2023 8:06:53 PM, 3/18/2023 7:37:20 PM, 3/20/2023 8:31:51 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 19.623152060356965 (T) = (0 -993.6746354566045) / Math.Sqrt((44116.33755433282 / (299)) + (541.3754648717687 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.24933344668875168 = (1323.7230712803557 - 993.6746354566045) / 1323.7230712803557 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Invoke.Property_Get_int ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 127.25185108159117 < 139.0688392277945. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/18/2023 3:39:51 AM, 3/18/2023 7:37:20 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 22.5633722431653 (T) = (0 -127.86363396787233) / Math.Sqrt((400.19809964237714 / (299)) + (2.8612644933478495 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.18913705816806928 = (157.68834333333598 - 127.86363396787233) / 157.68834333333598 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Invoke.Ctor0_NoParams ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 441.1527767361111 < 457.5919396964765. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 17.4031855024759 (T) = (0 -443.1528952588575) / Math.Sqrt((248.85087213909264 / (299)) + (287.0418722130634 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.10949096281105568 = (497.63997528621513 - 443.1528952588575) / 497.63997528621513 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7f0c19acfcfe1d89d1aaee814e6fd00b3ff1c8a2
Compare ef43a7b596545602a8bfe47d7a2b27a8ab5516ae
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Text.Json.Node.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 286.64 μs 229.81 μs 0.80 0.04 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20Json400B).html>) 161.94 μs 140.70 μs 0.87 0.03 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Node.Tests.Perf_ParseThenWrite*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 229.81421629680997 < 268.58561590933084. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/16/2023 12:15:41 AM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 24.087069550806554 (T) = (0 -232641.95195297027) / Math.Sqrt((4417934243.713211 / (299)) + (124373158.78532091 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3202112185998417 = (342226.81856237544 - 232641.95195297027) / 342226.81856237544 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 140.6977069750719 < 153.58862331227687. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/15/2023 2:53:22 PM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 18.446471374173534 (T) = (0 -140743.36145386606) / Math.Sqrt((2234710205.2926464 / (299)) + (23374280.441554863 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.27749131378630926 = (194798.1583327837 - 140743.36145386606) / 194798.1583327837 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7f0c19acfcfe1d89d1aaee814e6fd00b3ff1c8a2
Compare ef43a7b596545602a8bfe47d7a2b27a8ab5516ae
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToStream(MyEventsListerViewModel).JsonNet.html>) 15.76 ms 12.65 ms 0.80 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 12.653072886639675 < 14.67661925391617. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/22/2023 4:57:34 AM, 3/15/2023 9:07:40 PM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 6.445150240281768 (T) = (0 -13684066.118374994) / Math.Sqrt((7508136189827.737 / (299)) + (17414777220048.63 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3033089164433533 = (19641511.770923026 - 13684066.118374994) / 19641511.770923026 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7f0c19acfcfe1d89d1aaee814e6fd00b3ff1c8a2
Compare ef43a7b596545602a8bfe47d7a2b27a8ab5516ae
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Collections.Tests.Add_Remove_SteadyState<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Add_Remove_SteadyState(Int32).ImmutableQueue(Count%3a%20512).html>) 442.24 ns 321.07 ns 0.73 0.10 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Add_Remove_SteadyState(Int32).Queue(Count%3a%20512).html>) 187.77 ns 82.91 ns 0.44 0.12 False
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Add_Remove_SteadyState(Int32).ConcurrentStack(Count%3a%20512).html>) 271.00 ns 237.21 ns 0.88 0.09 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Add_Remove_SteadyState<Int32>.ImmutableQueue(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 321.0677451881974 < 415.80260234041583. IsChangePoint: Marked as a change because one of 3/16/2023 12:15:41 AM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.145462755656668 (T) = (0 -366.8427781428601) / Math.Sqrt((4506.880152343162 / (299)) + (28951.223587855504 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.17920861824884682 = (446.93790200404317 - 366.8427781428601) / 446.93790200404317 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Add_Remove_SteadyState<Int32>.Queue(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 82.90500486996343 < 168.51536788901214. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/16/2023 9:46:36 PM, 3/18/2023 12:49:26 AM, 3/18/2023 5:00:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 81.91048574675571 (T) = (0 -83.02160682415553) / Math.Sqrt((339.98777705754065 / (299)) + (0.837204451755821 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.523677347394939 = (174.29699463189755 - 83.02160682415553) / 174.29699463189755 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Add_Remove_SteadyState<Int32>.ConcurrentStack(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 237.21128663951296 < 262.88114235579457. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/15/2023 2:53:22 PM, 3/16/2023 12:15:41 AM, 3/19/2023 2:30:51 PM, 3/20/2023 11:25:43 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 17.567249412948406 (T) = (0 -238.1837032708413) / Math.Sqrt((3472.510499786224 / (299)) + (1.2222809274409752 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2036443030813181 = (299.0921069472338 - 238.1837032708413) / 299.0921069472338 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7f0c19acfcfe1d89d1aaee814e6fd00b3ff1c8a2
Compare ef43a7b596545602a8bfe47d7a2b27a8ab5516ae
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Xml.Linq.Perf_XName

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateElementWithNamespaceImplicitOperator - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XName.CreateElementWithNamespaceImplicitOperator.html>) 1.26 μs 1.11 μs 0.88 0.09 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XName*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Xml.Linq.Perf_XName.CreateElementWithNamespaceImplicitOperator ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1119064357543644 < 1.2288815439968388. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/15/2023 2:53:22 PM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.43028852947223 (T) = (0 -1236.920951623574) / Math.Sqrt((88131.81472339328 / (299)) + (381536.09495607915 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.21071771046347687 = (1567.1464671403055 - 1236.920951623574) / 1567.1464671403055 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7f0c19acfcfe1d89d1aaee814e6fd00b3ff1c8a2
Compare 56223423e5f81264371b76ab9476229716df8d91
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in PerfLabTests.BlockCopyPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CallBlockCopy - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements%3a%20100).html>) 241.19 ns 90.99 ns 0.38 0.10 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 90.99429109454927 < 225.82578294076788. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/16/2023 12:15:41 AM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 41.70073062745968 (T) = (0 -86.1921078370347) / Math.Sqrt((4481.692285976923 / (299)) + (3.8887406712692774 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.6534050801450172 = (248.68254812591584 - 86.1921078370347) / 248.68254812591584 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7f0c19acfcfe1d89d1aaee814e6fd00b3ff1c8a2
Compare ef43a7b596545602a8bfe47d7a2b27a8ab5516ae
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArrayCopyToIntArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size%3a%20512).html>) 447.73 ns 239.85 ns 0.54 0.09 False
[BitArrayCopyToIntArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size%3a%204).html>) 387.62 ns 179.02 ns 0.46 0.12 False
[BitArrayBitArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayBitArrayCtor(Size%3a%20512).html>) 653.39 ns 417.76 ns 0.64 0.10 False
[BitArrayIntArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size%3a%20512).html>) 982.56 ns 783.43 ns 0.80 0.04 False
[BitArrayByteArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size%3a%204).html>) 179.51 ns 141.68 ns 0.79 0.18 False
[BitArrayBitArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayBitArrayCtor(Size%3a%204).html>) 615.99 ns 421.56 ns 0.68 0.09 False
[BitArrayOr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayOr(Size%3a%204).html>) 98.57 ns 62.00 ns 0.63 0.14 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 239.85047722753262 < 424.8096014502641. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/15/2023 4:14:49 PM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 6.093159224616666 (T) = (0 -296.4717189116974) / Math.Sqrt((7727.427765834591 / (299)) + (13813.174306967509 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.3495340236057032 = (455.7835915648006 - 296.4717189116974) / 455.7835915648006 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 179.01577662689567 < 371.55938620799054. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 10.379490716793525 (T) = (0 -203.04674306936033) / Math.Sqrt((2785.991853996564 / (299)) + (5895.729310035332 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.4840620656118746 = (393.5487769670645 - 203.04674306936033) / 393.5487769670645 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayBitArrayCtor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 417.76472673517424 < 601.3091457159413. IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 3/17/2023 5:15:23 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.5248301630367074 (T) = (0 -491.52867583441207) / Math.Sqrt((12305.96612911576 / (299)) + (63960.18434611183 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2656808597839429 = (669.3665586461367 - 491.52867583441207) / 669.3665586461367 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 783.429833010944 < 934.0137246213633. IsChangePoint: Marked as a change because one of 3/15/2023 9:07:40 PM, 3/16/2023 7:54:34 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.4462237757406147 (T) = (0 -913.827488495431) / Math.Sqrt((8075.593518774069 / (299)) + (77881.72083292175 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.13784038719332173 = (1059.9284342728058 - 913.827488495431) / 1059.9284342728058 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 141.67560355465744 < 171.368347906203. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/15/2023 10:57:55 AM, 3/15/2023 9:07:40 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.2575539190786142 (T) = (0 -167.10427329802366) / Math.Sqrt((2486.572780954876 / (299)) + (6302.433928756499 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.178990720824788 = (203.53518228916613 - 167.10427329802366) / 203.53518228916613 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayBitArrayCtor(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 421.56099333079635 < 586.1033117063415. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/15/2023 4:14:49 PM, 3/16/2023 2:58:09 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.242355569016182 (T) = (0 -495.9819959355757) / Math.Sqrt((7054.617683357328 / (299)) + (114112.65114470663 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.24196856950040493 = (654.3026792552511 - 495.9819959355757) / 654.3026792552511 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 62.001741541676985 < 94.05872236383722. IsChangePoint: Marked as a change because one of 3/15/2023 8:07:42 AM, 3/15/2023 4:14:49 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.6654764007008582 (T) = (0 -79.49381931189029) / Math.Sqrt((560.3593828698229 / (299)) + (1799.9600724062202 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.22041849092078003 = (101.96986253019534 - 79.49381931189029) / 101.96986253019534 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7f0c19acfcfe1d89d1aaee814e6fd00b3ff1c8a2
Compare ef43a7b596545602a8bfe47d7a2b27a8ab5516ae
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.IO.MemoryMappedFiles.Tests.Perf_MemoryMappedFile

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateNew - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.MemoryMappedFiles.Tests.Perf_MemoryMappedFile.CreateNew(capacity%3a%2010000).html>) 43.99 μs 35.85 μs 0.81 0.07 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.MemoryMappedFiles.Tests.Perf_MemoryMappedFile*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.MemoryMappedFiles.Tests.Perf_MemoryMappedFile.CreateNew(capacity: 10000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 35.84612478595891 < 41.88185145654099. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 2:23:18 PM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 5.021722005217579 (T) = (0 -39513.11803907917) / Math.Sqrt((8705335.263237203 / (299)) + (56864803.22788081 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.1737276738695891 = (47820.93843578975 - 39513.11803907917) / 47820.93843578975 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)