search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.95k stars 4.65k forks source link

[Perf] Windows/x64: 1 Regression in IndexOfAny #77906

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline a8ea8e67dc20689aa2e4c738a2f84ea3483ec872
Compare 129982ef61894433867f0ea593281fc069284ade
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAny - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.18362/System.Tests.Perf_String.IndexOfAny.html>) 7.79 ns 12.61 ns 1.62 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_String.IndexOfAny ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 12.611276363670004 > 8.183525362767785. IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM. IsRegressionStdDev: Marked as regression because -35.25211814755959 (T) = (0 -11.46909107970713) / Math.Sqrt((0.006840580727820924 / (25)) + (0.2897007455461371 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.47583294510231 = (7.771266468720857 - 11.46909107970713) / 7.771266468720857 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
dotnet-issue-labeler[bot] commented 1 year ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

kunalspathak commented 1 year ago

https://github.com/dotnet/runtime/pull/76803 @stephentoub

kunalspathak commented 1 year ago

There are other regressions/improvements in the linked issues.

stephentoub commented 1 year ago

76803 @stephentoub

That PR is adding new methods, not updating existing ones.

More likely is https://github.com/dotnet/runtime/pull/75754, which changes existing IndexOfAny methods. cc: @adamsitnik

kunalspathak commented 1 year ago

More likely is https://github.com/dotnet/runtime/pull/75754, which changes existing IndexOfAny methods.

Ah sorry, I missed that.

kunalspathak commented 1 year ago

https://github.com/dotnet/perf-autofiling-issues/issues/9392 is interesting one too:

image

Rob-Hague commented 1 year ago

It does appear to be #75754 (3e40074e89015f49e039b2fd2b06989c689ddb8e)

diff --git a/src/benchmarks/micro/libraries/System.Runtime/Perf.String.cs b/src/benchmarks/micro/libraries/System.Runtime/Perf.String.cs
index 4a4646e7..08419463 100644
--- a/src/benchmarks/micro/libraries/System.Runtime/Perf.String.cs
+++ b/src/benchmarks/micro/libraries/System.Runtime/Perf.String.cs
@@ -5,11 +5,14 @@
 using System.Collections.Generic;
 using System.Linq;
 using BenchmarkDotNet.Attributes;
+using BenchmarkDotNet.Diagnostics.Windows.Configs;
 using MicroBenchmarks;

 namespace System.Tests
 {
     [BenchmarkCategory(Categories.Runtime, Categories.Libraries)]
+    [InliningDiagnoser(false, true)]
     public class Perf_String
     {
         // the culture-specific methods are tested in Perf_StringCultureSpecific class
cd c:\src\runtime\

git worktree add ../runtime-3e40074 3e40074
git worktree add ../runtime-e4471c1 3e40074~

cd ..\runtime-3e40074\
.\build.cmd -c Release
cd ..\runtime-e4471c1\
.\build.cmd -c Release

cd ../

py -3.11 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String.IndexOf*' --corerun C:\src\runtime-3e40074\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\corerun.exe C:\src\runtime-e4471c1\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\corerun.exe

BenchmarkDotNet=v0.13.2.1950-nightly, OS=Windows 11 (10.0.22621.674)
AMD Ryzen 7 5700U with Radeon Graphics, 1 CPU, 16 logical and 8 physical cores
.NET SDK=8.0.100-alpha.1.22554.7
  [Host]     : .NET 8.0.0 (8.0.22.55109), X64 RyuJIT AVX2
  Job-DOWZQX : .NET 8.0.0 (42.42.42.42424), X64 RyuJIT AVX2
  Job-YWAYCP : .NET 8.0.0 (42.42.42.42424), X64 RyuJIT AVX2

PowerPlanMode=00000000-0000-0000-0000-000000000000  IterationTime=250.0000 ms  MaxIterationCount=20  
MinIterationCount=15  WarmupCount=1
Method Job Toolchain Mean Error StdDev Median Min Max Ratio Code Size Allocated Alloc Ratio
IndexOfAny Job-DOWZQX \runtime-3e40074[..] 8.467 ns 0.1468 ns 0.1373 ns 8.489 ns 8.198 ns 8.657 ns 1.00 380 B - NA
IndexOfAny Job-YWAYCP \runtime-e4471c1[..] 5.447 ns 0.0424 ns 0.0396 ns 5.439 ns 5.395 ns 5.516 ns 0.64 2,325 B - NA 

// * Diagnostic Output - InliningDiagnoser *
--------------------

--------------------
Perf_String.IndexOfAny: Job-DOWZQX(PowerPlanMode=00000000-0000-0000-0000-000000000000, Toolchain=\runtime-3e40074\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\corerun.exe, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.String.IndexOfAny - instance int32  (wchar[])
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ThrowHelper.ThrowArgumentNullException - void  (value class System.ExceptionArgument)
Fail Reason: does not return
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char]..ctor - instance void  (!0&,int32)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].op_Implicit - value class System.ReadOnlySpan`1<!0>  (!0[])
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char]..ctor - instance void  (!0[])
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.Runtime.InteropServices.MemoryMarshal.GetArrayDataReference - generic !!0&  (!!0[])
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.MemoryExtensions.IndexOfAny - generic int32  (value class System.ReadOnlySpan`1<!!0>,value class System.ReadOnlySpan`1<!!0>)
Fail Reason: inline exceeds budget
--------------------

--------------------
Perf_String.IndexOfAny: Job-YWAYCP(PowerPlanMode=00000000-0000-0000-0000-000000000000, Toolchain=\runtime-e4471c1\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\corerun.exe, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.String.IndexOfAny - instance int32  (wchar[])
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ThrowHelper.ThrowArgumentNullException - void  (value class System.ExceptionArgument)
Fail Reason: does not return
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char]..ctor - instance void  (!0&,int32)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].op_Implicit - value class System.ReadOnlySpan`1<!0>  (!0[])
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char]..ctor - instance void  (!0[])
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.Runtime.InteropServices.MemoryMarshal.GetArrayDataReference - generic !!0&  (!!0[])
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.MemoryExtensions.IndexOfAny - generic int32  (value class System.ReadOnlySpan`1<!!0>,value class System.ReadOnlySpan`1<!!0>)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.Runtime.CompilerServices.RuntimeHelpers.IsBitwiseEquatable - generic bool  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.Runtime.InteropServices.MemoryMarshal.GetReference - generic !!0&  (value class System.ReadOnlySpan`1<!!0>)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.Runtime.InteropServices.MemoryMarshal.GetReference - generic !!0&  (value class System.ReadOnlySpan`1<!!0>)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.SpanHelpers.IndexOfValueType - generic int32  (!!0&,!!0,int32)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.SpanHelpers.IndexOfAnyValueType - generic int32  (!!0&,!!0,!!0,int32)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.SpanHelpers.IndexOfAnyValueType - generic int32  (!!0&,!!0,!!0,!!0,int32)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.SpanHelpers.IndexOfAnyValueType - generic int32  (!!0&,!!0,!!0,!!0,!!0,int32)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.MemoryExtensions.IndexOfAnyProbabilistic - int32  (wchar&,int32,wchar&,int32)
Fail Reason: unprofitable inline
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.Runtime.InteropServices.MemoryMarshal.GetReference - generic !!0&  (value class System.ReadOnlySpan`1<!!0>)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.Runtime.InteropServices.MemoryMarshal.GetReference - generic !!0&  (value class System.ReadOnlySpan`1<!!0>)
--------------------
Inliner: System.Tests.Perf_String.IndexOfAny - instance int32  ()
Inlinee: System.ReadOnlySpan`1[System.Char].get_Length - instance int32  ()
--------------------
ghost commented 1 year ago

Tagging subscribers to this area: @dotnet/area-system-memory See info in area-owners.md if you want to be subscribed.

Issue Details
### Run Information Architecture | x64 -- | -- OS | Windows 10.0.18362 Baseline | [a8ea8e67dc20689aa2e4c738a2f84ea3483ec872](https://github.com/dotnet/runtime/commit/a8ea8e67dc20689aa2e4c738a2f84ea3483ec872) Compare | [129982ef61894433867f0ea593281fc069284ade](https://github.com/dotnet/runtime/commit/129982ef61894433867f0ea593281fc069284ade) Diff | [Diff](https://github.com/dotnet/runtime/compare/a8ea8e67dc20689aa2e4c738a2f84ea3483ec872...129982ef61894433867f0ea593281fc069284ade) ### Regressions in System.Tests.Perf_String Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- [IndexOfAny - Duration of single invocation]() | 7.79 ns | 12.61 ns | 1.62 | 0.06 | False | | | ![graph]() [Test Report]() ### Repro ```cmd git clone https://github.com/dotnet/performance.git py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*' ``` ### Related Issues #### Regressions * [[Perf] Alpine/x64: 4 Regressions on 10/26/2022 10:56:11 AM]() * [[Perf] Windows/x64: 13 Regressions on 10/26/2022 10:56:11 AM]() * [[Perf] Windows/x64: 14 Regressions on 10/26/2022 10:56:11 AM]() * [[Perf] Linux/x64: 4 Regressions on 10/26/2022 10:56:11 AM]() #### Improvements * [[Perf] Linux/x64: 1 Improvement on 10/26/2022 10:56:11 AM]() * [[Perf] Linux/arm64: 2 Improvements on 10/26/2022 10:56:11 AM]() * [[Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM]() * [[Perf] Windows/arm64: 2 Improvements on 10/26/2022 10:56:11 AM]()
### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_String.IndexOfAny ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 12.611276363670004 > 8.183525362767785. IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM. IsRegressionStdDev: Marked as regression because -35.25211814755959 (T) = (0 -11.46909107970713) / Math.Sqrt((0.006840580727820924 / (25)) + (0.2897007455461371 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.47583294510231 = (7.771266468720857 - 11.46909107970713) / 7.771266468720857 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Author: performanceautofiler[bot]
Assignees: -
Labels: `area-System.Memory`, `tenet-performance-benchmarks`
Milestone: -
Rob-Hague commented 1 year ago

The change at https://github.com/Rob-Hague/runtime/tree/indexofany-ros appears to solve the regression. It is not super pretty given the work on deduplication with generics. I did play with generics ala

https://github.com/dotnet/runtime/blob/e0294993ba0c384a9ab5a778c2ba96731511bf9d/src/libraries/System.Private.CoreLib/src/System/MemoryExtensions.cs#L706-L716

but I was just adding complexity/indirection when ultimately it needs to become "un-generic" at some point (the complication here is the extra 4- and 5-value specialisations which only exist as helpers in SpanHelpers, along with the different fallbacks per type).

Edit: the tricky bit is that there is no non-value type helpers for 4- and 5- value "between" the 3- value:

https://github.com/dotnet/runtime/blob/136657403c045ab99e0cb18ef2a39e0179e6b434/src/libraries/System.Private.CoreLib/src/System/SpanHelpers.T.cs#L493

and the (ref, count, ref, count) helper:

https://github.com/dotnet/runtime/blob/136657403c045ab99e0cb18ef2a39e0179e6b434/src/libraries/System.Private.CoreLib/src/System/SpanHelpers.T.cs#L598

so when you have a pattern like this:


case 5:
    // private helper in MemoryExtensions
    return IndexOfAny(span, values[0], values[1], values[2], values[3], values[4]);

the fallback in the private helper after an IsBitwiseEquatable branch is not so obvious. So I was unsure how to proceed there. I don't doubt there is a nicer solution ("I've tried nothing and I'm all out of ideas!") but I can open a PR with these changes if they look OK.

Wall of benchmarks below. I added "LastIndex.." variants to the benchmark project locally. The comparisons are:

Toolchain Description
\runtime-e4471c1\ e4471c1 as baseline (the parent of #75754 = 3e40074)
\runtime-3e40074\ #75754 = 3e40074
\runtime\ main = d3ab95d3be895a1950a46c559397780dbb3e9807 (arbitrarily)
\indexofany-ros\ The indexofany-ros branch on my fork linked above. On the same main commit
\no-inline-attr The same indexofany-ros branch but with [AggressiveInlining] not added to the extra methods. Its favourability swings around a bit but it is probably beneficial to have the annotation on balance (purely from a time perspective) 

py -3.11 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String.*IndexOf*' 'System.Memory.Span<*>.*IndexOfAnyF*' --corerun C:\src\runtime-e4471c1\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\corerun.exe C:\src\runtime-3e40074\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\corerun.exe C:\src\runtime\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\corerun.exe C:\src\indexofany-ros\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\corerun.exe C:\src\no-inline-attr\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\corerun.exe
``` ini BenchmarkDotNet=v0.13.2.1950-nightly, OS=Windows 11 (10.0.22621.674) AMD Ryzen 7 5700U with Radeon Graphics, 1 CPU, 16 logical and 8 physical cores .NET SDK=8.0.100-alpha.1.22558.8 [Host] : .NET 8.0.0 (8.0.22.55109), X64 RyuJIT AVX2 Job-WUVVPK : .NET 8.0.0 (42.42.42.42424), X64 RyuJIT AVX2 Job-UEQUIS : .NET 8.0.0 (42.42.42.42424), X64 RyuJIT AVX2 Job-BCWCCT : .NET 8.0.0 (42.42.42.42424), X64 RyuJIT AVX2 Job-ARTMBH : .NET 8.0.0 (42.42.42.42424), X64 RyuJIT AVX2 Job-TUBRHZ : .NET 8.0.0 (42.42.42.42424), X64 RyuJIT AVX2 PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250.0000 ms MaxIterationCount=20 MinIterationCount=15 WarmupCount=1 ``` String: | Method | Toolchain | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | |--------------- |------------------------------------------------------------------------------------------------------------------ |---------:|----------:|----------:|---------:|---------:|---------:|------:|--------:| | IndexOfAny | \runtime-e4471c1\ | 5.454 ns | 0.0322 ns | 0.0302 ns | 5.441 ns | 5.419 ns | 5.504 ns | 1.00 | 0.00 | | IndexOfAny | \runtime-3e40074\ | 8.687 ns | 0.2597 ns | 0.2991 ns | 8.862 ns | 8.090 ns | 8.911 ns | 1.59 | 0.06 | | IndexOfAny | \runtime\ | 8.726 ns | 0.2019 ns | 0.2073 ns | 8.848 ns | 8.233 ns | 8.939 ns | 1.60 | 0.04 | | IndexOfAny | \indexofany-ros\ | 5.663 ns | 0.0335 ns | 0.0313 ns | 5.660 ns | 5.600 ns | 5.722 ns | 1.04 | 0.01 | | IndexOfAny | \no-inline-attr\ | 5.513 ns | 0.0145 ns | 0.0136 ns | 5.509 ns | 5.494 ns | 5.535 ns | 1.01 | 0.00 | | | | | | | | | | | | | LastIndexOfAny | \runtime-e4471c1\ | 5.296 ns | 0.0289 ns | 0.0271 ns | 5.298 ns | 5.263 ns | 5.359 ns | 1.00 | 0.00 | | LastIndexOfAny | \runtime-3e40074\ | 7.554 ns | 0.1851 ns | 0.2132 ns | 7.648 ns | 7.034 ns | 7.683 ns | 1.43 | 0.04 | | LastIndexOfAny | \runtime\ | 8.502 ns | 0.1527 ns | 0.1428 ns | 8.582 ns | 8.142 ns | 8.604 ns | 1.61 | 0.03 | | LastIndexOfAny | \indexofany-ros\ | 5.870 ns | 0.0440 ns | 0.0411 ns | 5.866 ns | 5.803 ns | 5.948 ns | 1.11 | 0.01 | | LastIndexOfAny | \no-inline-attr\ | 5.296 ns | 0.0273 ns | 0.0255 ns | 5.286 ns | 5.267 ns | 5.361 ns | 1.00 | 0.01 | Byte: | Method | Toolchain | Size | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | |------------------------- |----------------------------------------------------------------------------------------------------------------- |----- |-----------:|-----------:|-----------:|-----------:|-----------:|-----------:|------:|--------:| | IndexOfAnyFourValues |\runtime-e4471c1\ | 512 | 379.200 ns | 3.2807 ns | 3.0687 ns | 378.880 ns | 374.190 ns | 384.144 ns | 1.00 | 0.00 | | IndexOfAnyFourValues |\runtime-3e40074\ | 512 | 11.723 ns | 0.1891 ns | 0.1769 ns | 11.725 ns | 11.495 ns | 12.102 ns | 0.03 | 0.00 | | IndexOfAnyFourValues | \runtime\ | 512 | 11.458 ns | 0.1797 ns | 0.1500 ns | 11.482 ns | 11.187 ns | 11.703 ns | 0.03 | 0.00 | | IndexOfAnyFourValues | \indexofany-ros\ | 512 | 10.916 ns | 0.1895 ns | 0.1773 ns | 11.008 ns | 10.557 ns | 11.092 ns | 0.03 | 0.00 | | IndexOfAnyFourValues | \no-inline-attr\ | 512 | 14.001 ns | 0.1159 ns | 0.1084 ns | 14.046 ns | 13.814 ns | 14.192 ns | 0.04 | 0.00 | | | | | | | | | | | | | | IndexOfAnyFiveValues |\runtime-e4471c1\ | 512 | 433.278 ns | 3.5219 ns | 3.1221 ns | 433.354 ns | 426.580 ns | 438.826 ns | 1.00 | 0.00 | | IndexOfAnyFiveValues |\runtime-3e40074\ | 512 | 13.088 ns | 0.2924 ns | 0.2592 ns | 13.150 ns | 12.550 ns | 13.456 ns | 0.03 | 0.00 | | IndexOfAnyFiveValues | \runtime\ | 512 | 12.631 ns | 0.0724 ns | 0.0642 ns | 12.636 ns | 12.518 ns | 12.728 ns | 0.03 | 0.00 | | IndexOfAnyFiveValues | \indexofany-ros\ | 512 | 12.909 ns | 0.0380 ns | 0.0337 ns | 12.915 ns | 12.858 ns | 12.965 ns | 0.03 | 0.00 | | IndexOfAnyFiveValues | \no-inline-attr\ | 512 | 14.852 ns | 0.1449 ns | 0.1356 ns | 14.869 ns | 14.605 ns | 15.048 ns | 0.03 | 0.00 | | | | | | | | | | | | | | LastIndexOfAnyFourValues |\runtime-e4471c1\ | 512 | 359.321 ns | 6.1414 ns | 5.7447 ns | 357.324 ns | 353.518 ns | 371.797 ns | 1.00 | 0.00 | | LastIndexOfAnyFourValues |\runtime-3e40074\ | 512 | 11.261 ns | 0.2822 ns | 0.3250 ns | 11.328 ns | 10.792 ns | 11.805 ns | 0.03 | 0.00 | | LastIndexOfAnyFourValues | \runtime\ | 512 | 10.348 ns | 0.0431 ns | 0.0403 ns | 10.352 ns | 10.276 ns | 10.408 ns | 0.03 | 0.00 | | LastIndexOfAnyFourValues | \indexofany-ros\ | 512 | 9.831 ns | 0.0526 ns | 0.0466 ns | 9.812 ns | 9.784 ns | 9.933 ns | 0.03 | 0.00 | | LastIndexOfAnyFourValues | \no-inline-attr\ | 512 | 13.947 ns | 0.1151 ns | 0.1076 ns | 13.975 ns | 13.716 ns | 14.113 ns | 0.04 | 0.00 | | | | | | | | | | | | | | LastIndexOfAnyFiveValues |\runtime-e4471c1\ | 512 | 608.571 ns | 73.2890 ns | 84.3998 ns | 615.926 ns | 436.815 ns | 737.138 ns | 1.00 | 0.00 | | LastIndexOfAnyFiveValues |\runtime-3e40074\ | 512 | 622.467 ns | 86.5060 ns | 99.6204 ns | 639.169 ns | 451.811 ns | 820.048 ns | 1.04 | 0.22 | | LastIndexOfAnyFiveValues | \runtime\ | 512 | 719.187 ns | 41.5436 ns | 47.8416 ns | 732.196 ns | 591.086 ns | 787.878 ns | 1.21 | 0.21 | | LastIndexOfAnyFiveValues | \indexofany-ros\ | 512 | 417.953 ns | 1.3865 ns | 1.2969 ns | 418.062 ns | 415.932 ns | 419.931 ns | 0.72 | 0.11 | | LastIndexOfAnyFiveValues | \no-inline-attr\ | 512 | 619.922 ns | 45.3201 ns | 52.1906 ns | 636.418 ns | 506.068 ns | 700.475 ns | 1.03 | 0.14 | Char: | Method | Toolchain | Size | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | |------------------------- |------------------------------------------------------------------------------------------------------------------- |----- |----------:|---------:|---------:|----------:|----------:|----------:|------:|--------:| | IndexOfAnyFourValues | \runtime-e4471c1\ | 512 | 16.99 ns | 0.323 ns | 0.302 ns | 17.00 ns | 16.25 ns | 17.42 ns | 1.00 | 0.00 | | IndexOfAnyFourValues | \runtime-3e40074\ | 512 | 17.19 ns | 0.121 ns | 0.095 ns | 17.21 ns | 17.03 ns | 17.32 ns | 1.01 | 0.01 | | IndexOfAnyFourValues | \runtime\ | 512 | 17.00 ns | 0.339 ns | 0.317 ns | 17.11 ns | 16.25 ns | 17.34 ns | 1.00 | 0.03 | | IndexOfAnyFourValues | \indexofany-ros\ | 512 | 17.53 ns | 0.446 ns | 0.514 ns | 17.68 ns | 16.69 ns | 18.36 ns | 1.03 | 0.04 | | IndexOfAnyFourValues | \no-inline-attr\ | 512 | 20.70 ns | 0.352 ns | 0.329 ns | 20.74 ns | 19.70 ns | 21.05 ns | 1.22 | 0.03 | | | | | | | | | | | | | | IndexOfAnyFiveValues | \runtime-e4471c1\ | 512 | 19.51 ns | 0.116 ns | 0.103 ns | 19.50 ns | 19.24 ns | 19.69 ns | 1.00 | 0.00 | | IndexOfAnyFiveValues | \runtime-3e40074\ | 512 | 18.68 ns | 0.402 ns | 0.413 ns | 18.71 ns | 17.73 ns | 19.22 ns | 0.95 | 0.02 | | IndexOfAnyFiveValues | \runtime\ | 512 | 18.51 ns | 0.393 ns | 0.386 ns | 18.32 ns | 18.01 ns | 19.36 ns | 0.95 | 0.02 | | IndexOfAnyFiveValues | \indexofany-ros\ | 512 | 19.28 ns | 0.404 ns | 0.415 ns | 19.36 ns | 18.26 ns | 19.71 ns | 0.99 | 0.02 | | IndexOfAnyFiveValues | \no-inline-attr\ | 512 | 20.89 ns | 0.226 ns | 0.201 ns | 20.96 ns | 20.47 ns | 21.12 ns | 1.07 | 0.01 | | | | | | | | | | | | | | LastIndexOfAnyFourValues | \runtime-e4471c1\ | 512 | 357.85 ns | 6.492 ns | 5.755 ns | 358.16 ns | 347.21 ns | 369.02 ns | 1.00 | 0.00 | | LastIndexOfAnyFourValues | \runtime-3e40074\ | 512 | 17.07 ns | 0.227 ns | 0.201 ns | 17.15 ns | 16.53 ns | 17.27 ns | 0.05 | 0.00 | | LastIndexOfAnyFourValues | \runtime\ | 512 | 17.57 ns | 0.369 ns | 0.345 ns | 17.59 ns | 16.92 ns | 18.12 ns | 0.05 | 0.00 | | LastIndexOfAnyFourValues | \indexofany-ros\ | 512 | 17.33 ns | 0.319 ns | 0.298 ns | 17.47 ns | 16.52 ns | 17.60 ns | 0.05 | 0.00 | | LastIndexOfAnyFourValues | \no-inline-attr\ | 512 | 18.33 ns | 0.395 ns | 0.369 ns | 18.36 ns | 17.44 ns | 18.83 ns | 0.05 | 0.00 | | | | | | | | | | | | | | LastIndexOfAnyFiveValues | \runtime-e4471c1\ | 512 | 438.51 ns | 8.271 ns | 6.906 ns | 437.43 ns | 427.85 ns | 448.78 ns | 1.00 | 0.00 | | LastIndexOfAnyFiveValues | \runtime-3e40074\ | 512 | 210.38 ns | 1.763 ns | 1.472 ns | 210.42 ns | 206.61 ns | 212.76 ns | 0.48 | 0.01 | | LastIndexOfAnyFiveValues | \runtime\ | 512 | 204.47 ns | 0.262 ns | 0.204 ns | 204.46 ns | 204.07 ns | 204.79 ns | 0.47 | 0.01 | | LastIndexOfAnyFiveValues | \indexofany-ros\ | 512 | 214.60 ns | 4.235 ns | 4.160 ns | 216.03 ns | 208.86 ns | 221.10 ns | 0.49 | 0.01 | | LastIndexOfAnyFiveValues | \no-inline-attr\ | 512 | 209.92 ns | 2.315 ns | 1.807 ns | 210.41 ns | 205.22 ns | 212.24 ns | 0.48 | 0.01 | int: | Method | Toolchain | Size | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | |------------------------- |------------------------------------------------------------------------------------------------------------------- |----- |---------:|---------:|---------:|---------:|---------:|---------:|------:|--------:| | IndexOfAnyFourValues | \runtime-e4471c1\ | 512 | 539.8 ns | 0.32 ns | 0.26 ns | 539.8 ns | 539.5 ns | 540.2 ns | 1.00 | 0.00 | | IndexOfAnyFourValues | \runtime-3e40074\ | 512 | 350.8 ns | 4.51 ns | 3.52 ns | 350.7 ns | 345.8 ns | 357.9 ns | 0.65 | 0.01 | | IndexOfAnyFourValues | \runtime\ | 512 | 365.6 ns | 0.54 ns | 0.50 ns | 365.7 ns | 364.9 ns | 366.5 ns | 0.68 | 0.00 | | IndexOfAnyFourValues | \indexofany-ros\ | 512 | 399.3 ns | 36.71 ns | 39.28 ns | 394.7 ns | 354.9 ns | 516.4 ns | 0.75 | 0.08 | | IndexOfAnyFourValues | \no-inline-attr\ | 512 | 372.9 ns | 2.37 ns | 2.10 ns | 372.1 ns | 369.9 ns | 377.0 ns | 0.69 | 0.00 | | | | | | | | | | | | | | IndexOfAnyFiveValues | \runtime-e4471c1\ | 512 | 427.5 ns | 0.33 ns | 0.31 ns | 427.6 ns | 426.5 ns | 427.8 ns | 1.00 | 0.00 | | IndexOfAnyFiveValues | \runtime-3e40074\ | 512 | 419.2 ns | 2.39 ns | 1.87 ns | 419.4 ns | 416.3 ns | 422.4 ns | 0.98 | 0.00 | | IndexOfAnyFiveValues | \runtime\ | 512 | 427.9 ns | 0.50 ns | 0.47 ns | 427.7 ns | 427.2 ns | 428.7 ns | 1.00 | 0.00 | | IndexOfAnyFiveValues | \indexofany-ros\ | 512 | 448.2 ns | 15.99 ns | 16.42 ns | 439.6 ns | 433.8 ns | 485.5 ns | 1.04 | 0.03 | | IndexOfAnyFiveValues | \no-inline-attr\ | 512 | 437.4 ns | 3.89 ns | 3.63 ns | 436.9 ns | 433.2 ns | 443.9 ns | 1.02 | 0.01 | | | | | | | | | | | | | | LastIndexOfAnyFourValues | \runtime-e4471c1\ | 512 | 423.6 ns | 0.30 ns | 0.27 ns | 423.6 ns | 423.1 ns | 424.1 ns | 1.00 | 0.00 | | LastIndexOfAnyFourValues | \runtime-3e40074\ | 512 | 357.3 ns | 1.48 ns | 1.38 ns | 357.0 ns | 355.2 ns | 359.7 ns | 0.84 | 0.00 | | LastIndexOfAnyFourValues | \runtime\ | 512 | 424.1 ns | 0.51 ns | 0.47 ns | 424.1 ns | 423.6 ns | 425.0 ns | 1.00 | 0.00 | | LastIndexOfAnyFourValues | \indexofany-ros\ | 512 | 365.8 ns | 4.18 ns | 3.91 ns | 365.7 ns | 360.5 ns | 373.9 ns | 0.86 | 0.01 | | LastIndexOfAnyFourValues | \no-inline-attr\ | 512 | 380.6 ns | 23.16 ns | 26.67 ns | 365.9 ns | 357.9 ns | 449.1 ns | 0.90 | 0.07 | | | | | | | | | | | | | | LastIndexOfAnyFiveValues | \runtime-e4471c1\ | 512 | 540.8 ns | 0.79 ns | 0.74 ns | 540.8 ns | 539.8 ns | 542.1 ns | 1.00 | 0.00 | | LastIndexOfAnyFiveValues | \runtime-3e40074\ | 512 | 422.2 ns | 0.13 ns | 0.11 ns | 422.3 ns | 422.1 ns | 422.4 ns | 0.78 | 0.00 | | LastIndexOfAnyFiveValues | \runtime\ | 512 | 539.2 ns | 0.25 ns | 0.23 ns | 539.2 ns | 538.9 ns | 539.6 ns | 1.00 | 0.00 | | LastIndexOfAnyFiveValues | \indexofany-ros\ | 512 | 554.3 ns | 10.67 ns | 10.48 ns | 550.8 ns | 545.8 ns | 583.0 ns | 1.03 | 0.02 | | LastIndexOfAnyFiveValues | \no-inline-attr\ | 512 | 429.4 ns | 3.77 ns | 3.15 ns | 429.6 ns | 424.9 ns | 434.2 ns | 0.79 | 0.01 |
tannergooding commented 1 year ago

We're currently sitting at 8ns and this is nearly a year old now. I don't think there is really anything actionable for us to do here at this point.