dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 1 Regression on 7/10/2023 5:20:40 AM #19716

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline bd634023d2c09950721e29f75b70b4f3a9819d77
Compare 36481bfd079093fe7d29ef024b1fb801e6c787fe
Diff Diff
Configs CompilationMode:wasm, RunKind:micro

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[ReadStructFieldByFieldLE - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompilationMode=wasm_RunKind=micro/System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldLE.html>) 178.94 ns 234.31 ns 1.31 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'
### Payloads [Baseline]() [Compare]() ### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldLE #### ETL Files #### Histogram ```log ``` #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 234.31350462057435 > 185.8998751624048. IsChangePoint: Marked as a change because one of 3/21/2023 7:06:30 PM, 4/11/2023 3:40:00 PM, 4/17/2023 6:40:18 PM, 7/6/2023 7:42:33 PM, 7/10/2023 9:23:13 PM falls between 7/1/2023 10:14:52 PM and 7/10/2023 9:23:13 PM. IsRegressionStdDev: Marked as regression because -26.61851533651389 (T) = (0 -232.36726008706114) / Math.Sqrt((9.118886778264347 / (30)) + (7.575735568460207 / (2))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (2) - 2, .025) and -0.3016125839749256 = (178.52259800488952 - 232.36726008706114) / 178.52259800488952 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 year ago

range is https://github.com/dotnet/runtime/compare/0224f86a886e0aaf72acb6a4f5573236ba929292...53a10ea1a692e8ecd850fc4e62502ed6d9232411