search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 19 Regressions on 9/16/2023 11:57:54 AM #22010

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 9f4c5f7766ba6e6899bc998e17ca91a4fcc2d8d5
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
12.53 ns 15.13 ns 1.21 0.60 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 0) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.129537154757413 > 13.419661949915149. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.14393721381641 (T) = (0 -15.280339081394565) / Math.Sqrt((0.4494529730158871 / (20)) + (1.67744354436121 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.15840683254635904 = (13.190822647174823 - 15.280339081394565) / 13.190822647174823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 9f4c5f7766ba6e6899bc998e17ca91a4fcc2d8d5
Compare dc3d344c4803aaa74c2df444632b95554762bfa1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.13 ns 5.43 ns 1.31 0.81 False
3.46 ns 5.50 ns 1.59 0.83 False
3.35 ns 5.04 ns 1.50 0.82 False
4.93 ns 7.17 ns 1.45 0.82 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.BitwiseAndOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.426055379339407 > 3.031878809829962. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.958310068283993 (T) = (0 -5.003830466077067) / Math.Sqrt((0.5329987822508517 / (18)) + (0.6523164262820781 / (20))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (20) - 2, .025) and -0.5311325911671211 = (3.2680582301907948 - 5.003830466077067) / 3.2680582301907948 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.5043391506789705 > 3.900083593411578. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.630610001707102 (T) = (0 -6.341031310973321) / Math.Sqrt((0.6430799466749195 / (19)) + (1.837522626856246 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.6072753779460934 = (3.9452052821691352 - 6.341031310973321) / 3.9452052821691352 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.MultiplyOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.0410029685509175 > 3.2494447059647293. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.624331055116101 (T) = (0 -5.533153940466297) / Math.Sqrt((2.06453354386462 / (18)) + (0.8737595584745212 / (18))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (18) - 2, .025) and -0.509812511978866 = (3.664795394505082 - 5.533153940466297) / 3.664795394505082 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.ExclusiveOrOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.165420871466344 > 4.15317195611387. IsChangePoint: Marked as a change because one of 9/16/2023 7:32:07 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.671556408159693 (T) = (0 -5.979581647955499) / Math.Sqrt((0.8719914375149987 / (23)) + (2.5217267367018383 / (17))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (17) - 2, .025) and -0.6930075133820629 = (3.53192859493594 - 5.979581647955499) / 3.53192859493594 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 9f4c5f7766ba6e6899bc998e17ca91a4fcc2d8d5
Compare dc3d344c4803aaa74c2df444632b95554762bfa1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.71 ns 7.09 ns 2.62 0.85 False
2.95 ns 5.07 ns 1.72 0.83 False
3.72 ns 5.43 ns 1.46 0.87 False
2.86 ns 5.33 ns 1.87 0.82 False
2.96 ns 4.59 ns 1.55 0.88 False
2.89 ns 5.17 ns 1.79 0.81 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Double>.DivideBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.089099124670202 > 3.752312394326321. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.856403077337776 (T) = (0 -5.455988468730626) / Math.Sqrt((1.225621670725966 / (19)) + (0.6707739346214859 / (20))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (20) - 2, .025) and -0.3864069838774293 = (3.9353440455641735 - 5.455988468730626) / 3.9353440455641735 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.SubtractionOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.0710488940018505 > 4.57429850360922. IsChangePoint: Marked as a change because one of 9/16/2023 1:27:38 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.0129339728235305 (T) = (0 -5.691001419718089) / Math.Sqrt((1.2395640578236542 / (19)) + (2.034812453419376 / (20))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (20) - 2, .025) and -0.40477463896342214 = (4.05118462554069 - 5.691001419718089) / 4.05118462554069 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseOrBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.434321082548839 > 3.7997067864810394. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -2.555375208598287 (T) = (0 -5.253717499208555) / Math.Sqrt((2.314954681175469 / (19)) + (1.1729292100073654 / (17))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (17) - 2, .025) and -0.2698086659309339 = (4.137408760993847 - 5.253717499208555) / 4.137408760993847 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.333651251879192 > 4.412448851818375. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.0948182956542585 (T) = (0 -5.776453674662792) / Math.Sqrt((2.2451950821526285 / (19)) + (1.0359361616262799 / (17))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (17) - 2, .025) and -0.4285808006717938 = (4.0434910450612245 - 5.776453674662792) / 4.0434910450612245 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.586688233057645 > 3.4854766053316286. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.274856387675397 (T) = (0 -6.166010473560824) / Math.Sqrt((1.1198663342929214 / (19)) + (2.1202884658007566 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.5462514078412288 = (3.9877153497110727 - 6.166010473560824) / 3.9877153497110727 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Double>.BitwiseOrOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.1679579664932005 > 3.077443010405179. IsChangePoint: Marked as a change because one of 9/15/2023 4:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -2.5385140160293798 (T) = (0 -5.675004791335877) / Math.Sqrt((3.3953208876931242 / (16)) + (2.1089327705118737 / (24))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (24) - 2, .025) and -0.3245682378341503 = (4.284418597123606 - 5.675004791335877) / 4.284418597123606 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 9f4c5f7766ba6e6899bc998e17ca91a4fcc2d8d5
Compare dc3d344c4803aaa74c2df444632b95554762bfa1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CopyTo<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
997.43 ns 1.21 ΞΌs 1.21 0.25 False
964.21 ns 1.19 ΞΌs 1.24 0.25 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CopyTo&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CopyTo<Int32>.Array(Size: 2048) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2096212492467107 > 1044.3236693531558. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.431142114439263 (T) = (0 -1183.201382258081) / Math.Sqrt((9690.370327421735 / (19)) + (6719.682100391812 / (20))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (20) - 2, .025) and -0.12224490183997602 = (1054.316558104353 - 1183.201382258081) / 1054.316558104353 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CopyTo<Int32>.List(Size: 2048) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1914538680355546 > 1013.3102412262764. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.849876051308141 (T) = (0 -1156.036522174635) / Math.Sqrt((6723.761309631967 / (21)) + (7191.330414138825 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.1570768991919344 = (999.1008574987314 - 1156.036522174635) / 999.1008574987314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 9f4c5f7766ba6e6899bc998e17ca91a4fcc2d8d5
Compare dc3d344c4803aaa74c2df444632b95554762bfa1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Half

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.71 ns 15.31 ns 1.58 0.62 False
10.47 ns 12.13 ns 1.16 0.73 False
10.11 ns 11.63 ns 1.15 0.74 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Half*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Half.SingleToHalf(value: 6.097555E-05) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.311872040819695 > 10.5457006559303. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.29087360337369 (T) = (0 -12.203239892944577) / Math.Sqrt((1.340088089563127 / (20)) + (1.6840194093637337 / (20))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (20) - 2, .025) and -0.15838206981218592 = (10.534727885526705 - 12.203239892944577) / 10.534727885526705 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Half.SingleToHalf(value: 12345) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.128437524380935 > 11.234862848418572. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.637227419330408 (T) = (0 -12.566488586647054) / Math.Sqrt((1.6834584047860244 / (19)) + (2.062124994445711 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.1959473800430215 = (10.50755977758403 - 12.566488586647054) / 10.50755977758403 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Half.SingleToHalf(value: 65520) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.629292053853858 > 10.581199061785725. IsChangePoint: Marked as a change because one of 9/16/2023 7:32:07 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -7.133116169968404 (T) = (0 -11.77067259120768) / Math.Sqrt((0.607667865307056 / (20)) + (0.27648962862372767 / (16))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (16) - 2, .025) and -0.15247747181266075 = (10.213364581169916 - 11.77067259120768) / 10.213364581169916 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 9f4c5f7766ba6e6899bc998e17ca91a4fcc2d8d5
Compare dc3d344c4803aaa74c2df444632b95554762bfa1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.39 ΞΌs 6.03 ΞΌs 1.37 0.30 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'
### Payloads [Baseline]() [Compare]() ### System.MathBenchmarks.Single.Min #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.030454456347673 > 4.611231607668974. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.348799932334851 (T) = (0 -5614.431404946905) / Math.Sqrt((392464.08562255005 / (18)) + (523547.35482806416 / (20))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (20) - 2, .025) and -0.20433642555485126 = (4661.8463793123865 - 5614.431404946905) / 4661.8463793123865 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 9f4c5f7766ba6e6899bc998e17ca91a4fcc2d8d5
Compare dc3d344c4803aaa74c2df444632b95554762bfa1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Benchstone.BenchI.AddArray2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
21.87 ms 23.51 ms 1.08 0.39 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.AddArray2*'
### Payloads [Baseline]() [Compare]() ### Benchstone.BenchI.AddArray2.Test #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.509540006666665 > 21.285593553958332. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -8.044054297233982 (T) = (0 -23628098.384792794) / Math.Sqrt((1510547187407.6895 / (20)) + (1022950258164.9445 / (20))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (20) - 2, .025) and -0.13787517726155568 = (20765105.748819374 - 23628098.384792794) / 20765105.748819374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 9f4c5f7766ba6e6899bc998e17ca91a4fcc2d8d5
Compare dc3d344c4803aaa74c2df444632b95554762bfa1
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.35 ΞΌs 1.56 ΞΌs 1.15 0.26 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'
### Payloads [Baseline]() [Compare]() ### System.Buffers.Text.Tests.Base64Tests.ConvertTryFromBase64Chars(NumberOfBytes: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.555037722362836 > 1.4129411423174365. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.09863046192835 (T) = (0 -1518.896425335237) / Math.Sqrt((4575.4029539853245 / (20)) + (7891.243304949344 / (20))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (20) - 2, .025) and -0.11141402155727236 = (1366.634211800761 - 1518.896425335237) / 1366.634211800761 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)