search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Alpine/x64: 13 Regressions on 4/9/2023 11:24:15 AM #17041

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS alpine 3.15
Queue TigerAlpine
Baseline 0fc78e62bb0b8d824efe7421983702b97a60def6
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2012345).html>) 4.84 ns 24.62 ns 5.08 0.05 True
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%20-2147483648).html>) 11.39 ns 49.69 ns 4.36 0.07 True
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%20-9223372036854775808).html>) 21.69 ns 92.74 ns 4.28 0.02 True
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2018446744073709551615).html>) 21.17 ns 96.21 ns 4.54 0.00 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.615791292360118 > 5.0854480634519295. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -597.9257067123061 (T) = (0 -24.452980764699994) / Math.Sqrt((0.0014184848567328388 / (42)) + (0.01462768420156284 / (14))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (14) - 2, .025) and -4.077652897763685 = (4.815803926942239 - 24.452980764699994) / 4.815803926942239 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: -2147483648) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 49.68609659839048 > 11.984473629177993. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -684.9150488849963 (T) = (0 -49.63539372021156) / Math.Sqrt((0.10166308876723898 / (40)) + (0.007778566877493761 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -3.309289863608422 = (11.518230448914135 - 49.63539372021156) / 11.518230448914135 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 92.74210569853682 > 22.76125653026426. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -1440.898360699107 (T) = (0 -92.67864516674818) / Math.Sqrt((0.07053963132289197 / (37)) + (0.007234553119645117 / (14))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (14) - 2, .025) and -3.2613760764056985 = (21.748525242794095 - 92.67864516674818) / 21.748525242794095 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 96.20574048055006 > 22.22707995122859. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -1687.8332127326285 (T) = (0 -96.1873343647662) / Math.Sqrt((0.044655874873626836 / (41)) + (0.012368792169418929 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -3.5322276449940357 = (21.222970666755373 - 96.1873343647662) / 21.222970666755373 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS alpine 3.15
Queue TigerAlpine
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/MicroBenchmarks.Serializers.Json_ToString(CollectionsOfPrimitives).SystemTextJsonReflection.html>) 100.21 μs 112.20 μs 1.12 0.01 True
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/MicroBenchmarks.Serializers.Json_ToString(CollectionsOfPrimitives).SystemTextJsonSourceGen.html>) 97.79 μs 110.28 μs 1.13 0.00 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.20092064621315 > 105.55434104836704. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -41.7147846384471 (T) = (0 -112173.12174700559) / Math.Sqrt((574193.4596193656 / (43)) + (869446.9785409409 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.11377520462842897 = (100714.32842180044 - 112173.12174700559) / 100714.32842180044 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 110.28169243256744 > 102.37034706131709. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -37.70502205986756 (T) = (0 -109513.15853835817) / Math.Sqrt((1025701.4005327275 / (44)) + (986131.3406736719 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.11784119365378698 = (97968.44056211809 - 109513.15853835817) / 97968.44056211809 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS alpine 3.15
Queue TigerAlpine
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22G%22).html>) 152.79 ns 170.17 ns 1.11 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_DateTimeOffset.ToString(format: "G") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 170.1696521751888 > 160.48384931385883. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -14.013738971625607 (T) = (0 -166.21477609265315) / Math.Sqrt((5.27484414892749 / (41)) + (8.630397056555227 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.07848939001674046 = (154.1181374905071 - 166.21477609265315) / 154.1181374905071 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS alpine 3.15
Queue TigerAlpine
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/MicroBenchmarks.Serializers.Json_ToStream(CollectionsOfPrimitives).SystemTextJsonSourceGen.html>) 94.76 μs 107.82 μs 1.14 0.00 True
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/MicroBenchmarks.Serializers.Json_ToStream(CollectionsOfPrimitives).SystemTextJsonReflection.html>) 92.11 μs 102.19 μs 1.11 0.00 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 107.82060867172211 > 99.46028886145105. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -37.31828327545781 (T) = (0 -107591.74282450635) / Math.Sqrt((271516.5529437712 / (43)) + (1561465.2196128918 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.13516421252318395 = (94780.77412725778 - 107591.74282450635) / 94780.77412725778 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 102.19033043389274 > 97.15504139980158. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -41.06072669523795 (T) = (0 -102573.27641088869) / Math.Sqrt((403675.88196352945 / (44)) + (833881.3717920505 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.11725798986685063 = (91808.04911774484 - 102573.27641088869) / 91808.04911774484 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS alpine 3.15
Queue TigerAlpine
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20fr).html>) 129.45 ns 143.92 ns 1.11 0.02 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20).html>) 136.15 ns 149.94 ns 1.10 0.00 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20ja).html>) 134.78 ns 148.26 ns 1.10 0.01 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 143.92251921973278 > 136.022800959015. IsChangePoint: Marked as a change because one of 3/20/2023 10:25:17 PM, 4/6/2023 1:55:21 PM, 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -14.922244321943731 (T) = (0 -148.95510507275236) / Math.Sqrt((9.754904295976901 / (43)) + (13.68084019720291 / (13))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (13) - 2, .025) and -0.12778486867799035 = (132.07758785357726 - 148.95510507275236) / 132.07758785357726 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 149.9421635992813 > 143.12224682390078. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 4/9/2023 1:38:49 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -19.96771286839599 (T) = (0 -148.10752908251223) / Math.Sqrt((2.7284945038202406 / (42)) + (5.159030520983547 / (14))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (14) - 2, .025) and -0.09740858303246806 = (134.9611542797003 - 148.10752908251223) / 134.9611542797003 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ja) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 148.26230742585435 > 141.1085727985579. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -18.869590565253432 (T) = (0 -148.18275300862473) / Math.Sqrt((1.6147794202757353 / (41)) + (8.25172671560957 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.11231759342298452 = (133.21982308363508 - 148.18275300862473) / 133.21982308363508 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS alpine 3.15
Queue TigerAlpine
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_alpine 3.15/System.Tests.Perf_DateTime.ToString(format%3a%20%22G%22).html>) 144.31 ns 162.73 ns 1.13 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_DateTime.ToString(format: "G") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 162.7317061180974 > 154.59891001348555. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM falls between 4/3/2023 9:14:01 PM and 4/12/2023 4:44:56 PM. IsRegressionStdDev: Marked as regression because -7.841740681170022 (T) = (0 -164.57175461923865) / Math.Sqrt((7.710558230534384 / (43)) + (57.73220108506302 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.10968431175187705 = (148.30502051473223 - 164.57175461923865) / 148.30502051473223 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kunalspathak commented 1 year ago

Duplicate of https://github.com/dotnet/runtime/issues/84989