search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 21 Improvements on 11/14/2022 9:30:20 AM #10013

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1ad1022083def722cf43f057a46b644753543165
Compare 4276f0675b09a1e91c4879f8964a6150d7199fcf
Diff Diff

Improvements in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryAndCatchDeep_CaugtAtTheTop - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind%3a%20Hardware).html>) 51.00 μs 46.02 μs 0.90 0.22 False
[ThrowAndCatchDeep - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchDeep(kind%3a%20Software).html>) 14.95 μs 13.28 μs 0.89 0.44 False
[CatchAndThrowOtherDeep - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.CatchAndThrowOtherDeep(kind%3a%20Software).html>) 114.06 μs 105.71 μs 0.93 0.15 False
[ThrowAndCatchDeepRecursive - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchDeepRecursive(kind%3a%20ReflectionHardware).html>) 48.31 μs 39.25 μs 0.81 0.30 False
[ThrowAndCatchWhen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchWhen(kind%3a%20Hardware).html>) 16.80 μs 14.92 μs 0.89 0.26 False
[ThrowAndCatchDeepRecursive - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchDeepRecursive(kind%3a%20ReflectionSoftware).html>) 41.63 μs 34.37 μs 0.83 0.27 False
[ThrowAndCatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatch(kind%3a%20Hardware).html>) 17.33 μs 14.71 μs 0.85 0.30 False
[ThrowAndCatch_ManyCatchBlocks - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind%3a%20ReflectionSoftware).html>) 38.18 μs 34.12 μs 0.89 0.13 False
[ThrowAndCatchDeep - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchDeep(kind%3a%20Hardware).html>) 21.03 μs 17.64 μs 0.84 0.38 False
[MultipleNestedTryCatch_FirstCatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.MultipleNestedTryCatch_FirstCatches(kind%3a%20Hardware).html>) 16.77 μs 14.71 μs 0.88 0.25 False
[ThrowAndCatchWhenFinally - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchWhenFinally(kind%3a%20Hardware).html>) 17.11 μs 15.21 μs 0.89 0.30 False
[ThrowAndCatchWhenFinally - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchWhenFinally(kind%3a%20Software).html>) 11.78 μs 10.64 μs 0.90 0.19 False
[MultipleNestedTryCatch_FirstCatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.MultipleNestedTryCatch_FirstCatches(kind%3a%20Software).html>) 11.76 μs 10.56 μs 0.90 0.18 False
[ThrowAndCatchDeepRecursive - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchDeepRecursive(kind%3a%20Software).html>) 15.89 μs 13.56 μs 0.85 0.45 False
[ThrowAndCatchDeepRecursive - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchDeepRecursive(kind%3a%20Hardware).html>) 23.81 μs 18.13 μs 0.76 0.42 False
[TryAndFinallyDeep - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.TryAndFinallyDeep(kind%3a%20Hardware).html>) 32.26 μs 29.69 μs 0.92 0.25 False
[ThrowAndCatchDeep - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatchDeep(kind%3a%20ReflectionHardware).html>) 46.01 μs 39.35 μs 0.86 0.26 False
[TryAndFinallyDeep - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.TryAndFinallyDeep(kind%3a%20Software).html>) 27.77 μs 25.14 μs 0.91 0.25 False
[ThrowAndCatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatch(kind%3a%20ReflectionSoftware).html>) 35.77 μs 30.46 μs 0.85 0.18 False
[TryAndCatchDeep_CaugtAtTheTop - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind%3a%20Software).html>) 45.78 μs 41.36 μs 0.90 0.28 False
[ThrowAndCatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatch(kind%3a%20ReflectionHardware).html>) 39.86 μs 35.08 μs 0.88 0.19 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind: Hardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 46.018835759803906 < 49.49694847924506. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 35.010953886780726 (T) = (0 -45886.42344682547) / Math.Sqrt((3979993.911465411 / (299)) + (751478.338738296 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.12767112929673455 = (52602.20656211018 - 45886.42344682547) / 52602.20656211018 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchDeep(kind: Software) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.28343994691949 < 15.250895767851887. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 33.5545312076467 (T) = (0 -13442.574761390277) / Math.Sqrt((1087361.3484804153 / (299)) + (54771.94568574763 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.1543680149892959 = (15896.483339877594 - 13442.574761390277) / 15896.483339877594 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.CatchAndThrowOtherDeep(kind: Software) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 105.70538849999998 < 107.7994863849571. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 16.408078687418588 (T) = (0 -106157.51205846042) / Math.Sqrt((11930129.658275768 / (299)) + (3668628.2468869397 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.057923569415132325 = (112684.60669645968 - 106157.51205846042) / 112684.60669645968 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: ReflectionHardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 39.25353110416667 < 45.267787143775486. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 30.350837578695074 (T) = (0 -39444.944588049766) / Math.Sqrt((4211921.508399276 / (299)) + (1393564.1643664667 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.15592194738832235 = (46731.39464532033 - 39444.944588049766) / 46731.39464532033 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchWhen(kind: Hardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.92431013556619 < 15.797896393227191. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 22.2922010527882 (T) = (0 -14793.72081909809) / Math.Sqrt((382669.03030474024 / (299)) + (137977.68830406567 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.10126858829855169 = (16460.669591030663 - 14793.72081909809) / 16460.669591030663 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: ReflectionSoftware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 34.36694141941392 < 39.53411674423077. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 34.09437662238433 (T) = (0 -34922.734879605596) / Math.Sqrt((2868167.584202447 / (299)) + (848567.6622809098 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.15648803632175806 = (41401.58810234361 - 34922.734879605596) / 41401.58810234361 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatch(kind: Hardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.709534868217053 < 16.30988611861504. IsChangePoint: Marked as a change because one of 11/14/2022 1:59:26 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 20.86697076435429 (T) = (0 -14691.058235256638) / Math.Sqrt((416633.01503602177 / (299)) + (99629.40400507174 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09086718462921116 = (16159.419159526109 - 14691.058235256638) / 16159.419159526109 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: ReflectionSoftware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 34.12239120471014 < 36.02292056207299. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 21.036076891753783 (T) = (0 -34285.34582839933) / Math.Sqrt((1249128.5311489732 / (299)) + (1375898.6962066493 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.11910415108455587 = (38920.99828897063 - 34285.34582839933) / 38920.99828897063 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchDeep(kind: Hardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 17.6447957464146 < 20.54204979051622. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 34.642903537025965 (T) = (0 -18214.92088179008) / Math.Sqrt((1194855.8254098252 / (299)) + (128506.94803851563 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.14547336605528108 = (21315.802408293854 - 18214.92088179008) / 21315.802408293854 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.MultipleNestedTryCatch_FirstCatches(kind: Hardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.705920417451349 < 15.955750336208839. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 21.657325872773747 (T) = (0 -15089.869740889322) / Math.Sqrt((407556.0884164111 / (299)) + (98735.0622075201 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08824884582580078 = (16550.42570750204 - 15089.869740889322) / 16550.42570750204 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchWhenFinally(kind: Hardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.210437180746561 < 16.18367332588355. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 19.825715550695797 (T) = (0 -15057.813168622091) / Math.Sqrt((404881.62800971436 / (299)) + (150765.21962647687 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.0930050041233106 = (16601.870172467057 - 15057.813168622091) / 16601.870172467057 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchWhenFinally(kind: Software) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.635401127118644 < 11.0809619106955. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 16.48730618039822 (T) = (0 -10591.773579790288) / Math.Sqrt((136286.1832208687 / (299)) + (91291.38059352514 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.08217975945065488 = (11540.139465055563 - 10591.773579790288) / 11540.139465055563 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.MultipleNestedTryCatch_FirstCatches(kind: Software) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.56117513841808 < 10.963487630881769. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 20.336819329118384 (T) = (0 -10446.106501858454) / Math.Sqrt((119059.89604659546 / (299)) + (36096.7612155508 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.07145859909133745 = (11250.016953079297 - 10446.106501858454) / 11250.016953079297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: Software) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.562049830740424 < 15.750448662078297. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 30.7617137377759 (T) = (0 -13367.361087091873) / Math.Sqrt((1424099.5784225967 / (299)) + (95146.19116743047 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.17011421847062577 = (16107.470912994222 - 13367.361087091873) / 16107.470912994222 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: Hardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.126370411349747 < 21.352337872492928. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 32.19403495003075 (T) = (0 -17964.816665300983) / Math.Sqrt((1978992.9856072485 / (299)) + (142176.64163814185 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.15944954604427897 = (21372.680938730828 - 17964.816665300983) / 21372.680938730828 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.TryAndFinallyDeep(kind: Hardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 29.694982329800897 < 31.20529400230404. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 22.621682801195814 (T) = (0 -30180.343829275163) / Math.Sqrt((1744472.6986395004 / (299)) + (514171.0930987046 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.10090844115112466 = (33567.5977960639 - 30180.343829275163) / 33567.5977960639 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatchDeep(kind: ReflectionHardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 39.34795390413607 < 43.49580996695368. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 33.618920380048394 (T) = (0 -39472.806334717694) / Math.Sqrt((3417672.806368074 / (299)) + (1125564.7319021844 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.15530891702191799 = (46730.464107127234 - 39472.806334717694) / 46730.464107127234 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.TryAndFinallyDeep(kind: Software) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 25.14192412601626 < 26.59009251154761. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 28.117897811273693 (T) = (0 -25259.08437614761) / Math.Sqrt((1210391.9218789455 / (299)) + (149189.54711311872 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.09411425289582119 = (27883.300357570104 - 25259.08437614761) / 27883.300357570104 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatch(kind: ReflectionSoftware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 30.45814725728156 < 33.721440186443004. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 24.498374136181784 (T) = (0 -31468.313879273315) / Math.Sqrt((1079187.17855449 / (299)) + (1030171.2337757939 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.12839244446406956 = (36103.76445156468 - 31468.313879273315) / 36103.76445156468 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind: Software) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 41.356360682624114 < 43.4211970579116. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 49.453852581809166 (T) = (0 -40558.99566535684) / Math.Sqrt((2879618.386015798 / (299)) + (216688.15910454068 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.13584135471526956 = (46934.66400720103 - 40558.99566535684) / 46934.66400720103 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Exceptions.Handling.ThrowAndCatch(kind: ReflectionHardware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 35.08142670200893 < 38.38796660046097. IsChangePoint: Marked as a change because one of 11/14/2022 8:46:21 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 24.965308808053056 (T) = (0 -35909.64423177633) / Math.Sqrt((1868879.3580510772 / (299)) + (1409183.177001029 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.13650928470948906 = (41586.60144909024 - 35909.64423177633) / 41586.60144909024 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
lewing commented 1 year ago

looks like https://github.com/dotnet/runtime/pull/78182 or possibly a v8 change