search

DrewScoggins / performance-2

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
0 stars 0 forks source link

[Perf] Windows/x86: 18 Regressions on 7/13/2023 8:36:41 PM #8873

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline b9c99648783d0c28afdee394a93516c0bd228165
Compare 9409e2d5019dc46a7a8fd1c59b19a95ddaa3106e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.98 μs 4.22 μs 1.42 0.01 False
1.26 μs 1.35 μs 1.07 0.02 False
2.63 μs 3.65 μs 1.39 0.01 False
1.25 μs 1.35 μs 1.08 0.00 False
2.96 μs 4.28 μs 1.45 0.02 False
2.59 μs 3.65 μs 1.41 0.01 False
2.65 μs 3.69 μs 1.39 0.01 False
2.74 μs 3.61 μs 1.32 0.01 False
2.62 μs 3.68 μs 1.40 0.02 False
1.25 μs 1.35 μs 1.08 0.02 False
2.59 μs 3.62 μs 1.40 0.01 False
2.74 μs 3.73 μs 1.36 0.01 False
2.59 μs 3.66 μs 1.41 0.01 False
2.94 μs 4.34 μs 1.47 0.02 False
2.95 μs 4.30 μs 1.46 0.02 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.221032685026011 > 3.1199247816702913. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -43.417931050890715 (T) = (0 -4220.103928281795) / Math.Sqrt((1022.0555612499182 / (26)) + (3088.4464512521176 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.41456431515962633 = (2983.32418190938 - 4220.103928281795) / 2983.32418190938 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3542568201676322 > 1.322866679354083. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/23/2023 6:04:39 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -26.121967568850394 (T) = (0 -1348.7496938620386) / Math.Sqrt((147.58273093090529 / (26)) + (17.701297563908188 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.06559350139140965 = (1265.7262756397208 - 1348.7496938620386) / 1265.7262756397208 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6549649911376174 > 2.7885127936137644. IsChangePoint: Marked as a change because one of 5/21/2023 8:41:51 PM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -42.20511300364644 (T) = (0 -3597.707361904456) / Math.Sqrt((491.71383748329436 / (26)) + (1913.3078225257448 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.3542619542983146 = (2656.5815797199593 - 3597.707361904456) / 2656.5815797199593 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.348898505269549 > 1.3191252742658384. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/22/2023 7:55:14 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -19.90520899112545 (T) = (0 -1355.224877478427) / Math.Sqrt((121.24152150739151 / (26)) + (66.05285323494945 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.07248929621603269 = (1263.6255506324817 - 1355.224877478427) / 1263.6255506324817 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.280520597786353 > 3.1163364323516998. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -72.9837759225606 (T) = (0 -4285.441296132726) / Math.Sqrt((560.2379218113845 / (26)) + (1203.2273198574296 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.4404488043272994 = (2975.0736598612125 - 4285.441296132726) / 2975.0736598612125 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.649829343347415 > 2.752705519990284. IsChangePoint: Marked as a change because one of 5/22/2023 3:21:21 PM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -58.43750574371381 (T) = (0 -3667.713153814142) / Math.Sqrt((651.2754314911177 / (26)) + (1217.4592864507827 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.4068246011224522 = (2607.0863069126117 - 3667.713153814142) / 2607.0863069126117 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6864281701919177 > 2.7855616145202746. IsChangePoint: Marked as a change because one of 5/22/2023 6:29:00 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -59.740597244942116 (T) = (0 -3659.332899305602) / Math.Sqrt((737.9463363256745 / (26)) + (1018.6359428004648 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.3786605669070771 = (2654.2667478442822 - 3659.332899305602) / 2654.2667478442822 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6104696120980617 > 2.808553857597249. IsChangePoint: Marked as a change because one of 5/22/2023 6:29:00 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -48.241072632746324 (T) = (0 -3640.705447225808) / Math.Sqrt((866.3117415603025 / (26)) + (1501.3628682495091 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.3658645407918276 = (2665.495251172708 - 3640.705447225808) / 2665.495251172708 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6815186630289163 > 2.7397593497165578. IsChangePoint: Marked as a change because one of 5/22/2023 7:40:07 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -41.632993552810774 (T) = (0 -3629.6907554647487) / Math.Sqrt((569.196186274239 / (25)) + (2290.3145174255733 / (4))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (4) - 2, .025) and -0.3886338879183884 = (2613.85724995217 - 3629.6907554647487) / 2613.85724995217 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3522436135895985 > 1.3159405123479906. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/6/2023 4:13:54 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -22.145017989660484 (T) = (0 -1347.6254532693551) / Math.Sqrt((281.32211373266585 / (25)) + (11.065940348710193 / (4))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (4) - 2, .025) and -0.06556161497482402 = (1264.709083295193 - 1347.6254532693551) / 1264.709083295193 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.619761144130343 > 2.7321033487213833. IsChangePoint: Marked as a change because one of 5/22/2023 3:21:21 PM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -88.15810321850243 (T) = (0 -3635.802944745347) / Math.Sqrt((758.9255155277872 / (26)) + (434.25970838993265 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.3977934066062044 = (2601.1017991370804 - 3635.802944745347) / 2601.1017991370804 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.730855512553339 > 2.7802071802239836. IsChangePoint: Marked as a change because one of 5/22/2023 3:21:21 PM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -27.989737244802637 (T) = (0 -3680.1576594288845) / Math.Sqrt((738.1921288508116 / (25)) + (5233.260125108206 / (4))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (4) - 2, .025) and -0.3853980321471069 = (2656.39013051385 - 3680.1576594288845) / 2656.39013051385 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.658579300583254 > 2.7204734865649973. IsChangePoint: Marked as a change because one of 5/22/2023 3:21:21 PM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -65.53552611235062 (T) = (0 -3648.142582608122) / Math.Sqrt((350.5388818160767 / (26)) + (961.2379883857758 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.4009191729817155 = (2604.1063988320025 - 3648.142582608122) / 2604.1063988320025 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.341914712375919 > 3.127782857341907. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -55.515262836470754 (T) = (0 -4303.26954161677) / Math.Sqrt((802.4857751363348 / (26)) + (2150.207041399331 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.44419411012028787 = (2979.703013231613 - 4303.26954161677) / 2979.703013231613 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.3033987647106 > 3.10771156206294. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -93.20382530818611 (T) = (0 -4280.543606914826) / Math.Sqrt((600.0659796792543 / (26)) + (691.9110703410238 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.43859661415704837 = (2975.4995700605264 - 4280.543606914826) / 2975.4995700605264 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline b9c99648783d0c28afdee394a93516c0bd228165
Compare 9409e2d5019dc46a7a8fd1c59b19a95ddaa3106e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
851.01 ns 1.39 μs 1.63 0.11 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3874888838353865 > 913.4874475579367. IsChangePoint: Marked as a change because one of 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -104.32440144812017 (T) = (0 -1396.8907315809086) / Math.Sqrt((320.3383019563452 / (26)) + (51.68225119671493 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -0.6005480374466128 = (872.7577672766367 - 1396.8907315809086) / 872.7577672766367 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d9f6b98de38081b71fa9b5a463f302288546435d
Compare 9409e2d5019dc46a7a8fd1c59b19a95ddaa3106e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
54.22 ns 178.49 ns 3.29 0.12 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 178.48515055241694 > 57.97978827422276. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/13/2023 7:10:54 AM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -231.20289091650702 (T) = (0 -178.11683760649356) / Math.Sqrt((3.003218670840834 / (25)) + (0.6580712899463238 / (4))) is less than -2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (4) - 2, .025) and -2.252394854889551 = (54.76482578328955 - 178.11683760649356) / 54.76482578328955 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline b9c99648783d0c28afdee394a93516c0bd228165
Compare 9409e2d5019dc46a7a8fd1c59b19a95ddaa3106e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
48.94 ns 171.69 ns 3.51 0.06 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.GetService.Scoped #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 171.68749240066722 > 51.39347858221432. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/13/2023 3:01:51 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsRegressionStdDev: Marked as regression because -202.79979563782808 (T) = (0 -170.06611017186148) / Math.Sqrt((0.961208888321321 / (26)) + (1.2731490422003386 / (4))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (4) - 2, .025) and -2.4572968730486373 = (49.19048505715899 - 170.06611017186148) / 49.19048505715899 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)