search

joelverhagen / NCsvPerf

A test bench for various .NET CSV parsing libraries
https://www.joelverhagen.com/blog/2020/12/fastest-net-csv-parsers
MIT License
71 stars 14 forks source link

Add bench.ps1 and customize run and exporters #63

Closed nietras closed 10 months ago

nietras commented 10 months ago

@joelverhagen a few proposed changes to benchmark running now that args are forwarded. I do not know if you prefer the old way if so I can change to whatever you prefer. This change means csv file output is more directly usable e.g. in Excel. As output is sorted and units only defined in column names.

Changed run parameters since Sep and others are quite fast now and below 200 ms for a run, this can be tweaked to whatever you like. Currently, show casing how one can filter and only run Sep methods by --filter *.Sep* allowing for incremental updates.

Method,Job,AnalyzeLaunchVariance,EvaluateOverhead,MaxAbsoluteError,MaxRelativeError,MinInvokeCount,MinIterationTime,OutlierMode,Affinity,EnvironmentVariables,Jit,LargeAddressAware,Platform,PowerPlanMode,Runtime,AllowVeryLargeObjects,Concurrent,CpuGroups,Force,HeapAffinitizeMask,HeapCount,NoAffinitize,RetainVm,Server,Arguments,BuildConfiguration,Clock,EngineFactory,NuGetReferences,Toolchain,IsMutator,InvocationCount,IterationCount,IterationTime,LaunchCount,MaxIterationCount,MaxWarmupIterationCount,MemoryRandomization,MinIterationCount,MinWarmupIterationCount,RunStrategy,UnrollFactor,WarmupCount,LineCount,Mean [ms],Error [ms],StdDev [ms],Gen0,Gen1,Gen2,Allocated [KB]
Sep_MT,Job-LGJPSJ,False,Default,Default,Default,Default,Default,Default,11111111111111111111111111111111,Empty,RyuJit,Default,X64,8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c,.NET 8.0,False,True,False,True,Default,Default,False,False,False,Default,Default,Default,Default,Default,Default,Default,Default,Default,Default,Default,11,5,Default,5,3,Default,16,Default,1000000,309.0,35.37,25.58,16000.0000,15000.0000,1000.0000,268686.44
Sep,Job-LGJPSJ,False,Default,Default,Default,Default,Default,Default,11111111111111111111111111111111,Empty,RyuJit,Default,X64,8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c,.NET 8.0,False,True,False,True,Default,Default,False,False,False,Default,Default,Default,Default,Default,Default,Default,Default,Default,Default,Default,11,5,Default,5,3,Default,16,Default,1000000,682.5,24.98,18.06,18000.0000,17000.0000,3000.0000,266674.98

Example console output image

joelverhagen commented 10 months ago

Wow, that's an amazing delta in mean between multi and single threaded! Nice work!

nietras commented 10 months ago

Wow, that's an amazing delta in mean between multi and single threaded! Nice work!

Thanks, it should be close to 2x but with server gc is even more. For the floats bench I have in Sep I am seeing up to 7x speedup for multi-threaded. The parallel implementation is extremely efficient. 🚀 Note that those specific numbers are on dotnet 8.