dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 1 Regression on 8/22/2023 10:44:27 PM #21069

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 4a5695b6ea0e461e6527de057420872369345282
Compare c4f8c3b27a8fb2e452a57a6516050c8c2ee677f4
Diff Diff
Configs CompilationMode:wasm, RunKind:micro

Regressions in BilinearTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
44.27 μs 47.65 μs 1.08 0.23 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BilinearTest*'
### Payloads [Baseline]() [Compare]() ### BilinearTest.Interpol_Scalar #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.65296873088685 > 46.50631133168496. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsRegressionStdDev: Marked as regression because -7.88816259707777 (T) = (0 -47351.97329586121) / Math.Sqrt((373805.3451584789 / (11)) + (2569505.652442451 / (24))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (24) - 2, .025) and -0.0667382063920342 = (44389.49782816629 - 47351.97329586121) / 44389.49782816629 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 year ago

looks like noise, fixed or alternating result