[Perf] Linux/x64: 90 Regressions on 2/1/2023 6:08:01 AM

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	f70d8a9663d3d674c453e26d405dc13baa79fd16
Compare	4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80
Diff	Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(Int32).ImmutableList(Size%3a%20512).html>)	44.52 ms	50.07 ms	1.12	0.05	False
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(Int32).SortedSet(Size%3a%20512).html>)	622.65 μs	674.99 μs	1.08	0.07	False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(Int32).ImmutableSortedSet(Size%3a%20512).html>)	827.74 μs	893.49 μs	1.08	0.06	False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(Int32).LinkedList(Size%3a%20512).html>)	27.98 ms	33.33 ms	1.19	0.05	False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 50.072538442307696 > 47.08204081583334. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -19.560573979402264 (T) = (0 -50343736.79809124) / Math.Sqrt((1054678442820.3978 / (29)) + (1048003617175.7091 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.11377015710181639 = (45201190.27887458 - 50343736.79809124) / 45201190.27887458 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 674.9908859747025 > 654.682306175. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -13.258362811517882 (T) = (0 -676666.5838976173) / Math.Sqrt((285919041.0273229 / (29)) + (78795918.97171909 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.07385780139094102 = (630126.8035871678 - 676666.5838976173) / 630126.8035871678 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 893.4863608630951 > 870.490178582996. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -14.320945430167825 (T) = (0 -899792.0947261108) / Math.Sqrt((211506859.7459526 / (29)) + (357964331.8439371 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.07344406241391402 = (838229.1413515277 - 899792.0947261108) / 838229.1413515277 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<Int32>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 33.32608568571429 > 29.46462736463675. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -34.487731545476066 (T) = (0 -32234575.169746693) / Math.Sqrt((153684958498.49475 / (28)) + (247336356285.17484 / (32))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (32) - 2, .025) and -0.14025844074155863 = (28269534.359932628 - 32234575.169746693) / 28269534.359932628 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	f70d8a9663d3d674c453e26d405dc13baa79fd16
Compare	4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80
Diff	Diff

Regressions in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22Dynamic%22).html>)	69.81 μs	73.90 μs	1.06	0.02	False
[Singleton - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode%3a%20%22Dynamic%22).html>)	61.32 μs	64.76 μs	1.06	0.02	False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22Runtime%22).html>)	69.54 μs	73.77 μs	1.06	0.02	False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22ILEmit%22).html>)	68.71 μs	74.76 μs	1.09	0.01	False
[Singleton - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode%3a%20%22Runtime%22).html>)	61.73 μs	66.00 μs	1.07	0.02	False
[Singleton - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode%3a%20%22Expressions%22).html>)	61.42 μs	65.76 μs	1.07	0.01	False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22Expressions%22).html>)	69.63 μs	74.77 μs	1.07	0.02	False
[Transient - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode%3a%20%22ILEmit%22).html>)	59.94 μs	64.40 μs	1.07	0.01	False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'

### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 73.90090648924534 > 72.4321065195253. IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -26.9361342154968 (T) = (0 -74892.46160587772) / Math.Sqrt((241919.8955966163 / (29)) + (1335594.8534498746 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.08753242371123075 = (68864.57817074122 - 74892.46160587772) / 68864.57817074122 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 64.76011518595041 > 64.2655011507353. IsChangePoint: Marked as a change because one of 12/9/2022 8:25:19 AM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -21.121073217748314 (T) = (0 -65443.583745574135) / Math.Sqrt((413761.3154210043 / (29)) + (840014.1432163256 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.06947763206701192 = (61192.10143655771 - 65443.583745574135) / 61192.10143655771 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 73.77331466194968 > 72.9414033722345. IsChangePoint: Marked as a change because one of 12/9/2022 8:25:19 AM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -30.946103964377787 (T) = (0 -74748.13274604347) / Math.Sqrt((466940.4547200627 / (29)) + (619481.4631328171 / (31))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (31) - 2, .025) and -0.08535709765128582 = (68869.6217197073 - 74748.13274604347) / 68869.6217197073 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 74.7636064344099 > 72.53019250570458. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -33.13934417710488 (T) = (0 -75041.45137619032) / Math.Sqrt((391231.5708872438 / (28)) + (690280.4379706748 / (32))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (32) - 2, .025) and -0.09081916238662122 = (68793.66806502179 - 75041.45137619032) / 68793.66806502179 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 66.00336262746133 > 64.22534268072313. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -27.484664571677396 (T) = (0 -65720.94322068138) / Math.Sqrt((339067.9072452039 / (29)) + (541527.6805782609 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.07612819665103532 = (61071.66732105732 - 65720.94322068138) / 61071.66732105732 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 65.76103560126583 > 64.3680111752451. IsChangePoint: Marked as a change because one of 12/9/2022 9:29:01 AM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -24.155271216891418 (T) = (0 -65726.88609635168) / Math.Sqrt((472513.10852095083 / (29)) + (643864.2560031534 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.07541934724245604 = (61117.44805864403 - 65726.88609635168) / 61117.44805864403 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 74.7688396879984 > 72.63514985626529. IsChangePoint: Marked as a change because one of 12/9/2022 9:29:01 AM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -31.773789271300405 (T) = (0 -74654.79624124143) / Math.Sqrt((471696.1229750539 / (29)) + (471837.09832626075 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.08102115365932692 = (69059.51468991156 - 74654.79624124143) / 69059.51468991156 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 64.40435603908404 > 62.84141333914171. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -25.976608341424523 (T) = (0 -64446.10542681101) / Math.Sqrt((448324.87795699097 / (29)) + (465173.1882092711 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.07504933618103043 = (59947.11429314232 - 64446.10542681101) / 59947.11429314232 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	f70d8a9663d3d674c453e26d405dc13baa79fd16
Compare	4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80
Diff	Diff

Regressions in ByteMark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[BenchStringSort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/ByteMark.BenchStringSort.html>)	16.42 secs	18.00 secs	1.10	0.04	False
[BenchLUDecomp - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/ByteMark.BenchLUDecomp.html>)	16.35 secs	32.65 secs	2.00	0.08	False
[BenchNeuralJagged - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/ByteMark.BenchNeuralJagged.html>)	9.91 secs	19.20 secs	1.94	0.07	False
[BenchAssignJagged - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/ByteMark.BenchAssignJagged.html>)	12.30 secs	24.04 secs	1.95	0.07	False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'

### Payloads [Baseline]() [Compare]() ### Histogram #### ByteMark.BenchStringSort ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.9954309998 > 17.2627852674625. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -24.226519507127026 (T) = (0 -18113098066.543) / Math.Sqrt((31704301114896444 / (29)) + (1.0015145716962222E+17 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.0951914857519464 = (16538749891.857264 - 18113098066.543) / 16538749891.857264 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### ByteMark.BenchLUDecomp ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.65106323076923 > 17.171660932574998. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -6.106278650570077 (T) = (0 -28278902434.17351) / Math.Sqrt((2.095637882398067E+17 / (28)) + (1.1814519457034243E+20 / (32))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (32) - 2, .025) and -0.7103485144254238 = (16534000056.516876 - 28278902434.17351) / 16534000056.516876 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### ByteMark.BenchNeuralJagged ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.197401466466665 > 10.395686757574039. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -109.97622712817443 (T) = (0 -19446160702.643936) / Math.Sqrt((29851881385405092 / (29)) + (2.0657123932984083E+17 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.9580003920092433 = (9931642905.693624 - 19446160702.643936) / 9931642905.693624 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### ByteMark.BenchAssignJagged ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.037132266599997 > 12.645870687245. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -183.14290188687664 (T) = (0 -23534127355.313663) / Math.Sqrt((3280386871356595.5 / (29)) + (1.2169750407399117E+17 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.9492898492900758 = (12073180068.056429 - 23534127355.313663) / 12073180068.056429 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	f70d8a9663d3d674c453e26d405dc13baa79fd16
Compare	4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80
Diff	Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(Int32).SortedSet(Size%3a%20512).html>)	555.05 μs	598.28 μs	1.08	0.08	False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(Int32).LinkedList(Size%3a%20512).html>)	14.02 ms	16.17 ms	1.15	0.05	False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(Int32).ImmutableSortedSet(Size%3a%20512).html>)	724.32 μs	777.62 μs	1.07	0.04	False
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(Int32).ImmutableList(Size%3a%20512).html>)	21.96 ms	25.29 ms	1.15	0.06	False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 598.279190418956 > 580.1628075996767. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -10.245085854225042 (T) = (0 -610357.3022771743) / Math.Sqrt((163859104.0225122 / (29)) + (633143765.9781117 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.09248512346864586 = (558687.060505947 - 610357.3022771743) / 558687.060505947 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.171717948717948 > 14.873784309843751. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -23.27462004567526 (T) = (0 -16265513.935231177) / Math.Sqrt((122317302821.6351 / (29)) + (97700680231.73563 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.13897096233785874 = (14280885.53007926 - 16265513.935231177) / 14280885.53007926 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 777.6198656249999 > 760.9913771448865. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -24.52777664354669 (T) = (0 -786823.9185146072) / Math.Sqrt((30006899.47605454 / (29)) + (173794378.09925717 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.08609101222471287 = (724454.8658062303 - 786823.9185146072) / 724454.8658062303 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.291282034188033 > 23.242103864318185. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -22.299901039013033 (T) = (0 -25295008.988052886) / Math.Sqrt((243170623502.83917 / (29)) + (282070891053.1197 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.1307350475113601 = (22370412.09939038 - 25295008.988052886) / 22370412.09939038 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	f70d8a9663d3d674c453e26d405dc13baa79fd16
Compare	4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80
Diff	Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[LastWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input%3a%20IOrderedEnumerable).html>)	31.76 μs	33.93 μs	1.07	0.11	False
[AppendPrepend - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.AppendPrepend(input%3a%20IEnumerable).html>)	51.96 μs	57.03 μs	1.10	0.05	False
[Contains_ElementNotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input%3a%20IEnumerable).html>)	20.60 μs	23.03 μs	1.12	0.05	False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IOrderedEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 33.92958466127149 > 33.5912128718694. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -13.15272304202598 (T) = (0 -34379.63307996178) / Math.Sqrt((202549.96202928145 / (29)) + (620714.0946330416 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.0662564715992481 = (32243.305429505843 - 34379.63307996178) / 32243.305429505843 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.03156070120357 > 54.81125456135797. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -16.56776204722798 (T) = (0 -57173.17762032174) / Math.Sqrt((1623232.2743967052 / (29)) + (661524.2990765034 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.08722394611850103 = (52586.385559696086 - 57173.17762032174) / 52586.385559696086 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.034414705522977 > 21.637567007345474. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -19.308814902765402 (T) = (0 -23203.94888512119) / Math.Sqrt((281509.89176629606 / (29)) + (153271.87862099896 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.11134805651839814 = (20879.10151011908 - 23203.94888512119) / 20879.10151011908 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	f70d8a9663d3d674c453e26d405dc13baa79fd16
Compare	4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80
Diff	Diff

Regressions in System.Diagnostics.Perf_Activity

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[EnumerateActivityTagObjectsSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsSmall.html>)	660.46 ns	714.78 ns	1.08	0.04	False
[EnumerateActivityLinkTagsSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsSmall.html>)	655.18 ns	724.61 ns	1.11	0.07	False
[EnumerateActivityEventsSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall.html>)	677.06 ns	730.60 ns	1.08	0.04	False
[EnumerateActivityLinksSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall.html>)	676.96 ns	731.86 ns	1.08	0.05	False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsSmall ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 714.7758879448194 > 693.2077396674638. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -16.53878367914818 (T) = (0 -726.8656157280026) / Math.Sqrt((40.979566258510445 / (29)) + (447.76312981641934 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.09806598760364986 = (661.9507606407775 - 726.8656157280026) / 661.9507606407775 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsSmall ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 724.6133857770197 > 688.6709887566676. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -17.518589507420852 (T) = (0 -729.6653623041116) / Math.Sqrt((198.05642947511362 / (29)) + (247.101774158467 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.10081957739473026 = (662.8382863892956 - 729.6653623041116) / 662.8382863892956 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 730.6040026578661 > 709.507047363599. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -16.243261567416987 (T) = (0 -736.4023624596126) / Math.Sqrt((175.99769613613879 / (29)) + (187.80757095382396 / (31))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (31) - 2, .025) and -0.08320489062705758 = (679.8366300149512 - 736.4023624596126) / 679.8366300149512 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 731.8630310218351 > 709.8530682237977. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -15.102540304002744 (T) = (0 -740.4166909977868) / Math.Sqrt((239.00807811683555 / (29)) + (230.15452298476023 / (32))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (32) - 2, .025) and -0.08711399935736237 = (681.0846805721179 - 740.4166909977868) / 681.0846805721179 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	f70d8a9663d3d674c453e26d405dc13baa79fd16
Compare	4c25992c7b25c1ab21dc47e8dcba9f491b5fcb80
Diff	Diff

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryGetValueTrue(Int32%2c%20Int32).SortedList(Size%3a%20512).html>)	378.50 μs	403.07 μs	1.06	0.03	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryGetValueFalse(Int32%2c%20Int32).SortedList(Size%3a%20512).html>)	413.19 μs	450.04 μs	1.09	0.06	False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryGetValueFalse(Int32%2c%20Int32).SortedDictionary(Size%3a%20512).html>)	863.78 μs	945.91 μs	1.10	0.09	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndRemove(Int32).SortedSet(Size%3a%20512).html>)	1.67 ms	1.82 ms	1.09	0.04	False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndRemove(Int32).LinkedList(Size%3a%20512).html>)	220.75 μs	262.40 μs	1.19	0.06	False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndRemove(Int32).SortedDictionary(Size%3a%20512).html>)	2.13 ms	2.33 ms	1.09	0.08	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).Span(Size%3a%20512).html>)	10.70 μs	15.88 μs	1.48	0.13	False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).LinkedList(Size%3a%20512).html>)	103.92 μs	124.73 μs	1.20	0.12	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[BubbleSortArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.Sorting.BubbleSortArray(Size%3a%20512).html>)	2.71 ms	2.87 ms	1.06	0.06	False
[QuickSortArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.Sorting.QuickSortArray(Size%3a%20512).html>)	102.84 μs	115.28 μs	1.12	0.04	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[BitArrayOr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayOr(Size%3a%20512).html>)	1.80 μs	1.93 μs	1.07	0.06	False
[BitArrayXor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayXor(Size%3a%20512).html>)	1.81 μs	2.00 μs	1.11	0.10	False
[BitArrayXor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayXor(Size%3a%204).html>)	95.92 ns	101.78 ns	1.06	0.13	False
[BitArrayOr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayOr(Size%3a%204).html>)	95.14 ns	103.63 ns	1.09	0.15	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[Cos - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Cos.html>)	85.60 μs	99.38 μs	1.16	0.07	False
[Log2P1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Log2P1.html>)	106.47 μs	113.69 μs	1.07	0.06	False
[ScaleB - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.ScaleB.html>)	74.65 μs	79.24 μs	1.06	0.08	False
[Max - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Max.html>)	110.45 μs	128.57 μs	1.16	0.08	False
[Exp2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Exp2.html>)	137.63 μs	147.61 μs	1.07	0.08	False
[Tan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Tan.html>)	90.58 μs	120.33 μs	1.33	0.03	False
[Sin - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Sin.html>)	86.22 μs	91.85 μs	1.07	0.10	False
[Min - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Min.html>)	110.45 μs	129.06 μs	1.17	0.06	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrueComparer(Int32).SortedSet(Size%3a%20512).html>)	791.10 μs	859.42 μs	1.09	0.05	False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrueComparer(Int32).ImmutableSortedSet(Size%3a%20512).html>)	949.24 μs	1.02 ms	1.08	0.06	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[NewCustomConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).NewCustomConverter.html>)	748.85 μs	835.09 μs	1.12	0.02	False
[NewJsonSerializerContext - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).NewJsonSerializerContext.html>)	224.72 μs	254.86 μs	1.13	0.03	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).SortedDictionary(Size%3a%20512).html>)	901.46 μs	986.48 μs	1.09	0.11	False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).SortedList(Size%3a%20512).html>)	414.51 μs	444.64 μs	1.07	0.05	False

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).SortedDictionary(Size%3a%20512).html>)	1.03 ms	1.13 ms	1.10	0.10	False
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).SortedSet(Size%3a%20512).html>)	772.40 μs	858.25 μs	1.11	0.03	False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).Array(Size%3a%20512).html>)	7.94 μs	9.10 μs	1.15	0.09	False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).LinkedList(Size%3a%20512).html>)	101.75 μs	123.43 μs	1.21	0.06	False

Method	jiterp mean	no jiterp mean	ratio
List	32.35 us	37.13 us	87%
LinkedList	138.66 us	109.60 us	127%
HashSet	206.00 us	348.80 us	59%
Dictionary	220.18 us	304.76 us	72%
SortedList	2,800.26 us	3,164.29 us	88%
SortedSet	2,586.85 us	3,099.40 us	83%
SortedDictionary	2,864.75 us	3,498.72 us	82%
ConcurrentDictionary	668.23 us	726.46 us	92%
Stack	33.46 us	47.12 us	71%
ConcurrentStack	99.53 us	112.47 us	88%
Queue	59.20 us	69.43 us	85%
ConcurrentQueue	99.37 us	124.84 us	80%
ConcurrentBag	146.37 us	187.25 us	78%
ImmutableArray	542.50 us	593.12 us	91%
ImmutableList	1,626.56 us	1,638.02 us	99%
ImmutableDictionary	3,936.79 us	4,499.16 us	88%
ImmutableHashSet	3,836.35 us	4,416.19 us	87%
ImmutableSortedDictionary	4,936.30 us	5,612.19 us	88%
ImmutableSortedSet	4,751.74 us	5,324.15 us	89%
ImmutableStack	34.74 us	36.25 us	96%
ImmutableQueue	84.35 us	83.56 us	101%
Array	16.39 us	17.77 us	92%
Span	16.40 us	12.17 us	135%
ICollection	43.77 us	58.80 us	74%
IDictionary	223.08 us	313.40 us	71%

dotnet / perf-autofiling-issues