search

JoshClose / CsvHelper

Library to help reading and writing CSV files
http://joshclose.github.io/CsvHelper/
Other
4.73k stars 1.06k forks source link

Remove lock in hotpath #2223

Closed viceroypenguin closed 8 months ago

viceroypenguin commented 8 months ago

Using a lock in the hotpath of resolving every field of every record is fine for single-threaded operations, but when using a CsvReader across multiple threads, this creates a highly contentious serialization block.

This PR updates the hotpath to use a ConcurrentDictionary<,> instead. Given the expected use of being very read-heavy, a ConcurrentDictionary<,> will have high throughput across all threads, with minimal locking only on the initial encounter with each type.

viceroypenguin commented 8 months ago

Upon further testing, this had less of an impact than I expected:

// * Summary *

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22621.3007/22H2/2022Update/SunValley2)
12th Gen Intel Core i7-12700H, 1 CPU, 20 logical and 14 physical cores
.NET SDK 8.0.101
  [Host]        : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2
  .NET 6.0      : .NET 6.0.26 (6.0.2623.60508), X64 RyuJIT AVX2
  .NET 8.0      : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2
  .NET Core 3.1 : .NET Core 3.1.32 (CoreCLR 4.700.22.55902, CoreFX 4.700.22.56512), X64 RyuJIT AVX2
Method Job Runtime NumThreads Mean Error StdDev
ConcurrentDictionaryBased .NET 6.0 .NET 6.0 1 1,007.7 us 8.97 us 8.39 us
LockBased .NET 6.0 .NET 6.0 1 1,009.9 us 12.94 us 12.10 us
ConcurrentDictionaryBased .NET 8.0 .NET 8.0 1 875.3 us 10.21 us 9.55 us
LockBased .NET 8.0 .NET 8.0 1 872.7 us 13.09 us 12.25 us
ConcurrentDictionaryBased .NET Core 3.1 .NET Core 3.1 1 1,121.7 us 15.95 us 14.92 us
LockBased .NET Core 3.1 .NET Core 3.1 1 1,114.7 us 5.91 us 4.94 us
ConcurrentDictionaryBased .NET 6.0 .NET 6.0 4 1,098.9 us 7.76 us 7.26 us
LockBased .NET 6.0 .NET 6.0 4 1,106.3 us 9.71 us 8.10 us
ConcurrentDictionaryBased .NET 8.0 .NET 8.0 4 1,012.5 us 18.44 us 16.35 us
LockBased .NET 8.0 .NET 8.0 4 1,015.8 us 7.56 us 6.70 us
ConcurrentDictionaryBased .NET Core 3.1 .NET Core 3.1 4 1,242.6 us 11.67 us 9.74 us
LockBased .NET Core 3.1 .NET Core 3.1 4 1,247.9 us 8.58 us 7.61 us
ConcurrentDictionaryBased .NET 6.0 .NET 6.0 8 1,728.5 us 34.56 us 58.69 us
LockBased .NET 6.0 .NET 6.0 8 1,764.3 us 20.78 us 19.43 us
ConcurrentDictionaryBased .NET 8.0 .NET 8.0 8 1,675.4 us 27.30 us 25.53 us
LockBased .NET 8.0 .NET 8.0 8 1,693.4 us 33.19 us 32.60 us
ConcurrentDictionaryBased .NET Core 3.1 .NET Core 3.1 8 1,986.5 us 38.07 us 37.39 us
LockBased .NET Core 3.1 .NET Core 3.1 8 1,993.7 us 38.35 us 51.20 us
ConcurrentDictionaryBased .NET 6.0 .NET 6.0 16 2,393.3 us 34.91 us 29.15 us
LockBased .NET 6.0 .NET 6.0 16 2,460.1 us 42.93 us 40.16 us
ConcurrentDictionaryBased .NET 8.0 .NET 8.0 16 2,360.5 us 37.68 us 33.40 us
LockBased .NET 8.0 .NET 8.0 16 2,401.8 us 21.30 us 19.93 us
ConcurrentDictionaryBased .NET Core 3.1 .NET Core 3.1 16 2,778.3 us 52.33 us 48.95 us
LockBased .NET Core 3.1 .NET Core 3.1 16 2,795.5 us 39.16 us 36.63 us
JoshClose commented 8 months ago

Ok, I won't worry about it then.