search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.57k stars 4.55k forks source link

[Perf] Regressions from inliner changes #56012

Open DrewScoggins opened 2 years ago

DrewScoggins commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.html>) 32.94 ns 35.56 ns 1.08 0.03 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22string1%22%2c%20value%3a%20%22string2%22%2c%20comparisonType%3a%20InvariantCulture).html>) 42.75 ns 48.92 ns 1.14 0.08 True
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAA5AAAA%22%2c%20value%3a%20%225%22%2c%20comparisonType%3a%20InvariantCulture).html>) 41.68 ns 46.95 ns 1.13 0.11 True
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22foobardzsdzs%22%2c%20value%3a%20%22rddzs%22%2c%20comparisonType%3a%20InvariantCulture).html>) 52.62 ns 57.75 ns 1.10 0.07 True
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22ABCDE%22%2c%20value%3a%20%22c%22%2c%20comparisonType%3a%20InvariantCultureIgnoreCase).html>) 35.99 ns 43.20 ns 1.20 0.13 True
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22Hello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20Worldbbbbbbbbbbbbbbbbbb.html>) 19.85 ns 24.64 ns 1.24 0.06 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22Hello%20Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!%22%2c%20value%3a%20%22y%22%2c%20comparisonType%3a%20Ordinal).html>) 15.02 ns 18.69 ns 1.24 0.00 True
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%99%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%22%2c%20value%3a%20%22%e3%81%99%22%2c%20comparisonType%3a%20Ordinal).html>) 16.06 ns 23.18 ns 1.44 0.01 True

graph graph graph graph graph graph graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal) ```log ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture) ```log ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAA5AAAA", value: "5", comparisonType: InvariantCulture) ```log ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture) ```log ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase) ```log ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal) ```log ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal) ```log ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Formats.Cbor.Tests.Perf_CborReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SkipValue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding%3a%20(Text%20String%2c%20Strict)).html>) 63.45 ns 68.50 ns 1.08 0.01 False
[SkipValue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding%3a%20(Byte%20String%2c%20Strict)).html>) 50.81 ns 55.85 ns 1.10 0.01 False

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Formats.Cbor.Tests.Perf_CborReader*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Text String, Strict)) ```log ``` #### System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Byte String, Strict)) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Collections.Concurrent.Count<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.Concurrent.Count(String).Queue(Size%3a%20512).html>) 5.43 ns 6.78 ns 1.25 0.06 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Collections.Concurrent.Count&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.Count<String>.Queue(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Boolean.Parse(value%3a%20%22False%22).html>) 15.63 ns 16.90 ns 1.08 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Boolean.Parse(value%3a%20%22True%22).html>) 15.55 ns 16.68 ns 1.07 0.01 False

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Boolean*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Boolean.Parse(value: "False") ```log ``` #### System.Tests.Perf_Boolean.Parse(value: "True") ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Single.ToStringWithFormat(value%3a%20-3.4028235E%2b38%2c%20format%3a%20%22R%22).html>) 175.72 ns 188.64 ns 1.07 0.00 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Single.ToStringWithFormat(value%3a%203.4028235E%2b38%2c%20format%3a%20%22R%22).html>) 172.44 ns 185.01 ns 1.07 0.01 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Single.ToStringWithFormat(value%3a%20-3.4028235E%2b38%2c%20format%3a%20%22G%22).html>) 173.93 ns 185.70 ns 1.07 0.01 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Single.ToStringWithFormat(value%3a%203.4028235E%2b38%2c%20format%3a%20%22G%22).html>) 170.30 ns 186.85 ns 1.10 0.00 False

graph graph graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Single*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R") ```log ``` #### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R") ```log ``` #### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G") ```log ``` #### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G") ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Int16.ToString(value%3a%20-32768).html>) 21.02 ns 22.30 ns 1.06 0.17 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Int16*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int16.ToString(value: -32768) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Int32.ToString(value%3a%20-2147483648).html>) 28.26 ns 30.10 ns 1.07 0.20 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int32.ToString(value: -2147483648) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[PALong - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Net.NetworkInformation.Tests.PhysicalAddressTests.PALong.html>) 133.16 ns 143.10 ns 1.07 0.00 False
[PAMedium - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium.html>) 22.54 ns 23.99 ns 1.06 0.04 False

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.NetworkInformation.Tests.PhysicalAddressTests.PALong ```log ``` #### System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CertProp - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp.html>) 21.50 ns 27.51 ns 1.28 0.07 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_SByte.ToString(value%3a%20-128).html>) 17.88 ns 20.38 ns 1.14 0.23 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_SByte*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_SByte.ToString(value: -128) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Net.Primitives.Tests.IPAddressPerformanceTests.TryFormat(address%3a%201020%3a3040%3a5060%3a7080%3a9010%3a1112%3a1314%3a1516).html>) 106.68 ns 161.42 ns 1.51 0.03 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryFormat(address: 1020:3040:5060:7080:9010:1112:1314:1516) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

category:performance theme:benchmarks skill-level:intermediate cost:medium impact:small

dotnet-issue-labeler[bot] commented 2 years ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

DrewScoggins commented 2 years ago

Improvements from this change:

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetMethod20 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.GetMember.GetMethod20.html>) 2.25 ms 1.81 ms 0.80 0.01 False
[GetMethod4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.GetMember.GetMethod4.html>) 444.89 μs 358.61 μs 0.81 0.02 False
[GetMethod10 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.GetMember.GetMethod10.html>) 1.11 ms 905.43 μs 0.82 0.02 False
[GetMethod5 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.GetMember.GetMethod5.html>) 553.90 μs 447.70 μs 0.81 0.03 False
[GetMethod15 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.GetMember.GetMethod15.html>) 1.66 ms 1.35 ms 0.81 0.03 False
[GetMethod1 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.GetMember.GetMethod1.html>) 108.19 μs 89.30 μs 0.83 0.05 False
[GetMethod2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.GetMember.GetMethod2.html>) 220.10 μs 180.76 μs 0.82 0.01 False
[GetMethod3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.GetMember.GetMethod3.html>) 331.17 μs 268.91 μs 0.81 0.03 False
[GetMethod12 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/PerfLabTests.GetMember.GetMethod12.html>) 1.32 ms 1.08 ms 0.82 0.02 False

graph graph graph graph graph graph graph graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'PerfLabTests.GetMember*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.GetMember.GetMethod20 ```log ``` #### PerfLabTests.GetMember.GetMethod4 ```log ``` #### PerfLabTests.GetMember.GetMethod10 ```log ``` #### PerfLabTests.GetMember.GetMethod5 ```log ``` #### PerfLabTests.GetMember.GetMethod15 ```log ``` #### PerfLabTests.GetMember.GetMethod1 ```log ``` #### PerfLabTests.GetMember.GetMethod2 ```log ``` #### PerfLabTests.GetMember.GetMethod3 ```log ``` #### PerfLabTests.GetMember.GetMethod12 ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Int64.ToString(value%3a%209223372036854775807).html>) 45.69 ns 37.45 ns 0.82 0.03 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Int64.TryFormat(value%3a%209223372036854775807).html>) 39.43 ns 34.17 ns 0.87 0.02 True

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Int64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int64.ToString(value: 9223372036854775807) ```log ``` #### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 21.69 ns 17.27 ns 0.80 0.19 True
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 9.53 ns 6.79 ns 0.71 0.09 True
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(%2c%20None%2c%20False)).html>) 20.61 ns 16.43 ns 0.80 0.14 True
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 23.73 ns 17.25 ns 0.73 0.14 True
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 12.64 ns 10.55 ns 0.83 0.23 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 22.07 ns 17.06 ns 0.77 0.15 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(%2c%20None%2c%20False)).html>) 21.89 ns 17.41 ns 0.80 0.20 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 13.36 ns 10.53 ns 0.79 0.27 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 17.70 ns 14.14 ns 0.80 0.34 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 21.95 ns 18.00 ns 0.82 0.15 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 10.41 ns 8.23 ns 0.79 0.05 True
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 21.16 ns 16.37 ns 0.77 0.15 True
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 21.45 ns 16.49 ns 0.77 0.15 True
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 22.07 ns 17.58 ns 0.80 0.13 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreCase, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, Ordinal, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, IgnoreCase, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, None, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, None, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, Ordinal, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, Ordinal, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, IgnoreCase, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False)) ```log ``` #### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreCase, False)) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetDirectoryName - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.IO.Tests.Perf_Path.GetDirectoryName.html>) 66.89 ns 57.87 ns 0.87 0.09 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.IO.Tests.Perf_Path*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_Path.GetDirectoryName ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Guid.Parse.html>) 84.35 ns 74.00 ns 0.88 0.01 True
[ctor_str - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Guid.ctor_str.html>) 87.68 ns 77.27 ns 0.88 0.00 True

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Guid*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Guid.Parse ```log ``` #### System.Tests.Perf_Guid.ctor_str ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22Hello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20Worldbbbbbbbbbbbbbbbbbbb.html>) 35.93 ns 30.72 ns 0.86 0.10 True
[IndexOfString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22StrIng%22%2c%20value%3a%20%22string%22%2c%20comparisonType%3a%20OrdinalIgnoreCase).html>) 28.03 ns 24.42 ns 0.87 0.15 False

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase) ```log ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateScaleFromScalarXYZWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZWithCenterBenchmark.html>) 11.58 ns 8.79 ns 0.76 0.01 True
[CreateFromScalars - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_Matrix4x4.CreateFromScalars.html>) 16.25 ns 11.00 ns 0.68 0.01 True
[CreateRotationXWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark.html>) 23.77 ns 8.83 ns 0.37 0.04 True
[CreateRotationYWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark.html>) 23.47 ns 8.83 ns 0.38 0.08 True
[CreateScaleFromScalarXYZBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark.html>) 11.58 ns 8.79 ns 0.76 0.01 True
[CreateRotationZWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark.html>) 25.42 ns 8.85 ns 0.35 0.04 True
[CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark.html>) 10.68 ns 8.83 ns 0.83 0.00 True
[CreateOrthographicOffCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark.html>) 11.63 ns 8.48 ns 0.73 0.00 True

graph graph graph graph graph graph graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZWithCenterBenchmark ```log ``` #### System.Numerics.Tests.Perf_Matrix4x4.CreateFromScalars ```log ``` #### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark ```log ``` #### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark ```log ``` #### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark ```log ``` #### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark ```log ``` #### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark ```log ``` #### System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt64.ToString(value%3a%2018446744073709551615).html>) 45.54 ns 37.99 ns 0.83 0.03 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_UInt64.TryFormat(value%3a%2018446744073709551615).html>) 40.42 ns 35.47 ns 0.88 0.02 False

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_UInt64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615) ```log ``` #### System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Text.Json.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteByteArrayAsBase64_NoEscaping - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes%3a%20100).html>) 104.11 ns 96.00 ns 0.92 0.02 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Text.Json.Tests.Perf_Base64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 100) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Split - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_String.Split(s%3a%20%22A%20B%20C%20D%20E%20F%20G%20H%20I%20J%20K%20L%20M%20N%20O%20P%20Q%20R%20S%20T%20U%20V%20W%20X%20Y%20Z%22%2c%20arr%3a%20%5b%27%20%27%5d%2c%20options%3a%20None).html>) 418.44 ns 377.65 ns 0.90 0.03 False
[Interpolation_MultipleArgs - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_String.Interpolation_MultipleArgs(c%3a%20%271%27%2c%20s%3a%20%22Foo%22).html>) 253.23 ns 106.62 ns 0.42 0.03 True
[TrimEnd_CharArr - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_String.TrimEnd_CharArr(s%3a%20%22Test%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 5.04 ns 3.88 ns 0.77 0.16 False

graph graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: None) ```log ``` #### System.Tests.Perf_String.Interpolation_MultipleArgs(c: '1', s: "Foo") ```log ``` #### System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', ' ']) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Enum.Parse(text%3a%20%22Red%22).html>) 127.46 ns 116.24 ns 0.91 0.01 False
[EnumToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Enum.EnumToString(value%3a%20Yellow).html>) 20.66 ns 18.81 ns 0.91 0.12 False

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Enum*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Enum.Parse(text: "Red") ```log ``` #### System.Tests.Perf_Enum.EnumToString(value: Yellow) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Version.Parse3.html>) 42.20 ns 36.62 ns 0.87 0.03 False
[Parse4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Version.Parse4.html>) 66.95 ns 54.08 ns 0.81 0.03 True
[Parse2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Version.Parse2.html>) 38.59 ns 29.58 ns 0.77 0.03 True

graph graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Version*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Version.Parse3 ```log ``` #### System.Tests.Perf_Version.Parse4 ```log ``` #### System.Tests.Perf_Version.Parse2 ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22E%22).html>) 196.19 ns 184.22 ns 0.94 0.00 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_Double.ToStringWithFormat(value%3a%201.7976931348623157E%2b308%2c%20format%3a%20%22E%22).html>) 199.38 ns 184.03 ns 0.92 0.00 False

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Tests.Perf_Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E") ```log ``` #### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E") ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromUtf8Bytes.html>) 108.74 ns 100.05 ns 0.92 0.01 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Collections.CtorDefaultSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.CtorDefaultSize(Int32).Dictionary.html>) 5.95 ns 4.09 ns 0.69 0.37 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Collections.CtorDefaultSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorDefaultSize<Int32>.Dictionary ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 92547f59890b738e02f115277f272a381c9f3f3b
Compare 56d9474f530579c67b294cbbde0724e966dbb05b
Diff Diff

Improvemnts in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Collections.IterateForEach(Int32).ImmutableList(Size%3a%20512).html>) 10.66 μs 6.83 μs 0.64 0.08 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.ImmutableList(Size: 512) ```log ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
EgorBo commented 2 years ago

This issue has 11 regressions and 16 improvements. Almost all regressions have "U"-shape - it means my first PR https://github.com/dotnet/runtime/pull/52708 improved it and the second one https://github.com/dotnet/runtime/pull/55478 regressed it back. So overall it's a zero impact (Actually, in some benchmarks it's still an improvement after second PR). The second PR introduced a couple of "slowdown" mechanisms for the inliner like "if there are too many locals in the root method already - slow down inlining new calls" - I'd like to keep it, but it for sure has to be tuned, like changing linear dependency on number of locals to quadratic, etc.

Here are the "U" shapes:

image

And here is starting point: image

EgorBo commented 2 years ago

The only exception is the first one ReadOnlySpan.IndexOfString it suffers from two things: 1) That mechanism to slow-down if there are too many locals 2) if we inline a method that has a pinvoke - we have to import CORINFO_HELP_JIT_PINVOKE_BEGIN into prologue, even if the inline was invoked only under certain condition.

EgorBo commented 2 years ago

I'm going to move it to 7.0 since the overall it's an improvement (even some of the IndexOfString were actually improved). For 7.0 I'm going to revise those two reasons ^.