search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 41 Improvements on 3/2/2023 2:22:24 PM #13801

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToLower - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_String.ToLower(s%3a%20%22This%20is%20a%20much%20longer%20piece%20of%20text%20that%20might%20benefit%20more%20from%20vectorization.%22).html>) 1.86 μs 1.70 μs 0.91 0.16 False
[IndexerCheckBoundCheckHoist - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_String.IndexerCheckBoundCheckHoist.html>) 829.37 ns 172.23 ns 0.21 0.13 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.7018100200962545 < 1.7394079866361873. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 17.726194168760113 (T) = (0 -1730.0970043188709) / Math.Sqrt((473062.76220085646 / (299)) + (3519.9789656554476 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.29629619807396484 = (2458.5585577107877 - 1730.0970043188709) / 2458.5585577107877 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.IndexerCheckBoundCheckHoist ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 172.22937134350528 < 787.6147738159449. IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 627.6515900299424 (T) = (0 -172.41462471185898) / Math.Sqrt((337.9640914593712 / (299)) + (0.9053991680814424 / (44))) is greater than 1.9669451293272056 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (44) - 2, .975) and 0.7961416376269633 = (845.7569397931326 - 172.41462471185898) / 845.7569397931326 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.IO.Tests.StreamReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%200%2c%20%20%20%200%5d).html>) 365.59 ms 235.83 ms 0.65 0.12 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%209%2c%20%20%2032%5d).html>) 359.67 ms 226.51 ms 0.63 0.13 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>) 352.67 ms 230.34 ms 0.65 0.11 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>) 353.12 ms 229.56 ms 0.65 0.14 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>) 354.71 ms 238.91 ms 0.67 0.14 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%208%5d).html>) 370.27 ms 230.29 ms 0.62 0.13 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>) 354.06 ms 229.35 ms 0.65 0.13 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%201%5d).html>) 355.21 ms 229.62 ms 0.65 0.13 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 0, 0]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 235.82575005 < 336.4373480403847. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 29.224858350824846 (T) = (0 -229604995.24013206) / Math.Sqrt((17749385406351776 / (299)) + (11863457335196.057 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.49587744378175247 = (455454715.1441686 - 229604995.24013206) / 455454715.1441686 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 9, 32]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 226.50926706666667 < 339.7000863557692. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 28.847699008099028 (T) = (0 -228856546.81837606) / Math.Sqrt((18369005826171556 / (299)) + (15556994856121.688 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.49793632124362397 = (455831713.1907636 - 228856546.81837606) / 455831713.1907636 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 230.33661515384614 < 334.67582895881407. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 28.68017148869454 (T) = (0 -229266676.52812746) / Math.Sqrt((18133819536484304 / (299)) + (14398444221086.283 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4943626302141558 = (453421147.69964534 - 229266676.52812746) / 453421147.69964534 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 229.55584584615386 < 342.9387552628846. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 28.838413030038275 (T) = (0 -229710843.6085859) / Math.Sqrt((18137743342952044 / (299)) + (28733509067733.56 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.496166612347599 = (455926203.459675 - 229710843.6085859) / 455926203.459675 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 238.9078498 < 339.63980899826925. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 28.789496871938997 (T) = (0 -229693751.2456488) / Math.Sqrt((18142672256131176 / (299)) + (24257449439774.773 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.49551855568750464 = (455306639.78865314 - 229693751.2456488) / 455306639.78865314 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 8]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 230.292231 < 347.1622122565384. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 28.623251139661644 (T) = (0 -228566658.04324007) / Math.Sqrt((19149295408180220 / (299)) + (13087822208616.299 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5013167554682104 = (458340360.4382975 - 228566658.04324007) / 458340360.4382975 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 229.35192307692307 < 338.1975520692308. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 27.87215314974234 (T) = (0 -230140334.9186869) / Math.Sqrt((19051926312475004 / (299)) + (27852873230868.918 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4931900098506115 = (454095892.7247077 - 230140334.9186869) / 454095892.7247077 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 1]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 229.62284615384615 < 338.69424963685896. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 28.80351256659106 (T) = (0 -229262881.1100233) / Math.Sqrt((18183953009522856 / (299)) + (25104416218185.97 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4964423861859638 = (455286296.58391 - 229262881.1100233) / 455286296.58391 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 18.23 μs 14.20 μs 0.78 0.07 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 18.10 μs 14.66 μs 0.81 0.06 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromString(Mode%3a%20Reflection).html>) 29.55 μs 19.02 μs 0.64 0.06 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromReader(Mode%3a%20SourceGen).html>) 25.92 μs 19.12 μs 0.74 0.06 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromReader(Mode%3a%20Reflection).html>) 24.69 μs 18.97 μs 0.77 0.05 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromString(Mode%3a%20SourceGen).html>) 26.94 μs 19.11 μs 0.71 0.05 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;BinaryData&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.198567378246754 < 17.394571476061152. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 17.83165261424121 (T) = (0 -14054.287191693023) / Math.Sqrt((248981286.4717011 / (299)) + (73531.90334975957 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5368955290999954 = (30347.984255862822 - 14054.287191693023) / 30347.984255862822 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.662458290220295 < 17.320639047330552. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 17.417219656652975 (T) = (0 -14354.260225002896) / Math.Sqrt((253195766.28297696 / (299)) + (106698.04312055084 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5280150607451654 = (30412.538687495547 - 14354.260225002896) / 30412.538687495547 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 19.01590421995534 < 26.147095749452625. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 21.431879545469727 (T) = (0 -19081.42925525455) / Math.Sqrt((426716469.6813714 / (299)) + (74337.67996677352 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5731685549440287 = (44704.83483884924 - 19081.42925525455) / 44704.83483884924 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 19.117154012963493 < 23.76765234427239. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 15.484727597840854 (T) = (0 -19103.030259216386) / Math.Sqrt((786326115.7453218 / (299)) + (109660.85025514317 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5680998240580404 = (44230.19790986036 - 19103.030259216386) / 44230.19790986036 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.96606017783794 < 23.207846968546363. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 15.379307441014717 (T) = (0 -18878.75261121081) / Math.Sqrt((794844751.9896913 / (299)) + (62287.812025994754 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5705737345189049 = (43962.73383525005 - 18878.75261121081) / 43962.73383525005 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 19.107450683937373 < 26.076743285767865. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 21.508784498287035 (T) = (0 -19261.01014771609) / Math.Sqrt((426217884.2406733 / (299)) + (45919.62836219704 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5715358104965932 = (44953.60550444075 - 19261.01014771609) / 44953.60550444075 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetString.html>) 85.46 μs 75.87 μs 0.89 0.10 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Get.GetString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 75.86713155339807 < 81.27669225953917. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 23.276749810397636 (T) = (0 -75384.37137741252) / Math.Sqrt((287106714.1928012 / (299)) + (2002777.2965136243 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.23779689312862473 = (98903.25911533437 - 75384.37137741252) / 98903.25911533437 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyFourValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).IndexOfAnyFourValues(Size%3a%20512).html>) 1.30 μs 1.23 μs 0.94 0.21 False
[BinarySearch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).BinarySearch(Size%3a%2033).html>) 392.78 ns 356.49 ns 0.91 0.10 False
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).Reverse(Size%3a%20512).html>) 3.51 μs 796.10 ns 0.23 0.07 False
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).Reverse(Size%3a%2033).html>) 296.97 ns 104.58 ns 0.35 0.10 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.225584126506024 < 1.2262313942666239. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 7.8388177713715645 (T) = (0 -1245.528883216216) / Math.Sqrt((12844724.253479956 / (299)) + (1210.0411789464601 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5661595709422309 = (2870.9377913932603 - 1245.528883216216) / 2870.9377913932603 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.BinarySearch(Size: 33) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 356.4916122977185 < 370.449665379676. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/28/2023 5:58:27 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 10.434733775200266 (T) = (0 -354.62002060910214) / Math.Sqrt((1357.4136341727246 / (299)) + (32.23345159114055 / (44))) is greater than 1.9669451293272056 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (44) - 2, .975) and 0.06328908839858048 = (378.58000394469275 - 354.62002060910214) / 378.58000394469275 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.Reverse(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 796.098773662336 < 3.1556484555873996. IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 20.828471113530085 (T) = (0 -797.9508668372379) / Math.Sqrt((5841026.774742853 / (299)) + (8.242794567666875 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7848685498446023 = (3709.1316321293207 - 797.9508668372379) / 3709.1316321293207 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.Reverse(Size: 33) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 104.57705779914005 < 251.654663378167. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 22.591532300490748 (T) = (0 -105.7666013255449) / Math.Sqrt((25252.326665356126 / (299)) + (20.70342791246713 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6633269813169174 = (314.1522945297414 - 105.7666013255449) / 314.1522945297414 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SetEnvironmentVariable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Environment.SetEnvironmentVariable.html>) 972.73 ns 898.64 ns 0.92 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Environment*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Environment.SetEnvironmentVariable ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 898.6361814956225 < 923.1317468829923. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 19.275785602669202 (T) = (0 -916.315157623561) / Math.Sqrt((245617.60876144565 / (299)) + (454.34540693812244 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.37809163972491033 = (1473.392570600347 - 916.315157623561) / 1473.392570600347 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes%3a%201000).html>) 15.04 μs 10.71 μs 0.71 0.07 False
[ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes%3a%20100).html>) 2.39 μs 1.89 μs 0.79 0.09 False
[ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes%3a%20100).html>) 2.45 μs 1.89 μs 0.77 0.09 False
[ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes%3a%201000).html>) 14.62 μs 10.60 μs 0.72 0.13 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.708935959636952 < 14.409659945356543. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 16.94607720585693 (T) = (0 -10645.31754687366) / Math.Sqrt((190794948.52946824 / (299)) + (20614.263154551034 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.559906680390142 = (24188.773318146974 - 10645.31754687366) / 24188.773318146974 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.8856920168799738 < 2.287497859470613. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 17.042055938860162 (T) = (0 -1887.5727142153835) / Math.Sqrt((2704100.6174837737 / (299)) + (2020.1484038186334 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4628009578350833 = (3513.730602736091 - 1887.5727142153835) / 3513.730602736091 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.8913306195685173 < 2.335022296380105. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 16.713697298952283 (T) = (0 -1885.7199840568585) / Math.Sqrt((2818472.868882109 / (299)) + (1333.909742269218 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.463051025365898 = (3511.9165379575625 - 1885.7199840568585) / 3511.9165379575625 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.59951139167801 < 14.54225561919192. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 17.23276504549336 (T) = (0 -10661.667437392534) / Math.Sqrt((191914839.97220722 / (299)) + (90674.57579387781 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5647832007965419 = (24497.371096211624 - 10661.667437392534) / 24497.371096211624 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).Reverse(Size%3a%20512).html>) 1.04 μs 695.39 ns 0.67 0.12 False
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).Reverse(Size%3a%2033).html>) 120.64 ns 101.95 ns 0.85 0.13 False
[IndexOfAnyThreeValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).IndexOfAnyThreeValues(Size%3a%20512).html>) 785.82 ns 737.83 ns 0.94 0.17 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.Reverse(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 695.3942828701158 < 1.036998061202232. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 16.79973157773767 (T) = (0 -696.7241212747881) / Math.Sqrt((4335352.6816053875 / (299)) + (15.287183579329335 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7438210644136434 = (2719.6776334482265 - 696.7241212747881) / 2719.6776334482265 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.Reverse(Size: 33) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 101.94752166573782 < 111.47126457882784. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 17.174156223695892 (T) = (0 -104.34773937738906) / Math.Sqrt((21579.054926769753 / (299)) + (3.1532990507201593 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.58318348264886 = (250.344540184052 - 104.34773937738906) / 250.344540184052 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 737.8295761520656 < 752.3107853742312. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 8.164088578906943 (T) = (0 -756.5502252760833) / Math.Sqrt((5810016.730329346 / (299)) + (392.0361022490608 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6007538124029987 = (1894.946648907627 - 756.5502252760833) / 1894.946648907627 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>) 200.76 μs 162.68 μs 0.81 0.07 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>) 198.61 μs 159.01 μs 0.80 0.07 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>) 203.65 μs 158.12 μs 0.78 0.08 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>) 254.81 μs 217.07 μs 0.85 0.06 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 162.68018496691482 < 190.94931122915708. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 19.610096081567825 (T) = (0 -156915.2942671423) / Math.Sqrt((18804849009.092487 / (299)) + (3490444.349984575 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4979729667932064 = (312563.43560787133 - 156915.2942671423) / 312563.43560787133 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 159.0132236177565 < 189.11093192312595. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 19.71130244391816 (T) = (0 -156372.33221377258) / Math.Sqrt((17833409342.274868 / (299)) + (5610587.143923668 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4936423959263667 = (308817.9795380999 - 156372.33221377258) / 308817.9795380999 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 158.12495393041237 < 188.52433990004258. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 19.736978036590944 (T) = (0 -158748.29977820578) / Math.Sqrt((17577432337.319447 / (299)) + (10477893.963435454 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4887096733932238 = (310485.63119070337 - 158748.29977820578) / 310485.63119070337 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 217.06850954861113 < 244.11163092472543. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 18.417099293328707 (T) = (0 -220173.8161175708) / Math.Sqrt((19503858644.88386 / (299)) + (15841446.333382037 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4040761318976626 = (369466.34948301915 - 220173.8161175708) / 369466.34948301915 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Array.Reverse.html>) 1.73 μs 368.96 ns 0.21 0.07 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Array.Reverse ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 368.955850248817 < 1.6393127047391338. IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 13.146245827814393 (T) = (0 -360.19541490074454) / Math.Sqrt((4590940.560769779 / (299)) + (26.13022801690586 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8189264928748462 = (1989.2220602530545 - 360.19541490074454) / 1989.2220602530545 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Dictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20Reflection).html>) 389.99 μs 340.75 μs 0.87 0.06 False

graph_1.png>) Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Dictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 340.7516570785985 < 367.70752576289306. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 21.239624730638717 (T) = (0 -344828.9026885786) / Math.Sqrt((14193309384.421453 / (299)) + (35236771.47283956 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.3002692196114599 = (492802.2495982028 - 344828.9026885786) / 492802.2495982028 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 207.59 μs 179.98 μs 0.87 0.04 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromString(Mode%3a%20SourceGen).html>) 213.85 μs 191.64 μs 0.90 0.04 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromString(Mode%3a%20Reflection).html>) 214.90 μs 191.89 μs 0.89 0.05 False

graph_1.png>) graph_2.png>) graph_3.png>) Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 179.98364232427053 < 186.6662445901606. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 19.47337495730674 (T) = (0 -182038.2101264412) / Math.Sqrt((2610725513.614669 / (299)) + (9235376.407886678 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.24306947443373925 = (240495.26869095175 - 182038.2101264412) / 240495.26869095175 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 191.6439843441815 < 206.11363947606077. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 22.816431731857147 (T) = (0 -193011.1438839413) / Math.Sqrt((3948749470.287954 / (299)) + (10528551.972935338 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.3030161599585796 = (276923.41313461587 - 193011.1438839413) / 276923.41313461587 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 191.89059381163713 < 210.00767839199864. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 22.517838629595108 (T) = (0 -193035.3370036773) / Math.Sqrt((3958452549.9767356 / (299)) + (15250273.10298507 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.30157115011137003 = (276385.1135795125 - 193035.3370036773) / 276385.1135795125 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadAllLines - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_File.ReadAllLines.html>) 145.69 μs 135.97 μs 0.93 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_File.ReadAllLines ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 135.97358697916667 < 138.2059584406684. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 19.16084724255781 (T) = (0 -134065.33878222478) / Math.Sqrt((2288303502.0372634 / (299)) + (3183701.3506780304 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.28462517442867913 = (187405.72632697268 - 134065.33878222478) / 187405.72632697268 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 6a218ab799e56d4ebaffda550de06a9a73a88b14
Compare 90e540203bbf41fd901d09549b47445540aa34d5
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20Reflection).html>) 1.03 ms 952.77 μs 0.93 0.02 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 1.01 ms 923.56 μs 0.92 0.03 False

graph_1.png>) graph_2.png>) Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 952.7686887254902 < 0.9690802640201108. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 22.367212987641977 (T) = (0 -969169.4101118704) / Math.Sqrt((21781494839.511497 / (299)) + (263177651.79999992 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.17182995578708726 = (1170254.1246018654 - 969169.4101118704) / 1170254.1246018654 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 923.5644790723981 < 940.0885136944111. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/1/2023 3:09:21 PM, 3/2/2023 11:15:04 AM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM. IsImprovementStdDev: Marked as improvement because 21.809305387194843 (T) = (0 -943082.0187022422) / Math.Sqrt((16378137615.796309 / (299)) + (228771245.21502653 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15373490862902128 = (1114404.9640218725 - 943082.0187022422) / 1114404.9640218725 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kg commented 1 year ago

Probably https://github.com/dotnet/runtime/pull/82886