JoshClose / CsvHelper

Library to help reading and writing CSV files
http://joshclose.github.io/CsvHelper/
Other
4.65k stars 1.05k forks source link

Fetching ~2700 columns with GetRecord<dynamic> taking >7s #2191

Open Tak1za opened 10 months ago

Tak1za commented 10 months ago

Describe the bug When I am fetching 1 row with nearly 2700 columns from a csv file, it takes ~8 seconds to fetch that row. Is it expected to be this slow? If yes, are there any performance benchmarks that are done that can help me provide an explanation that it's the best this library can do?

To Reproduce Generate an excel with ~2700 columns and consume one row using GetRecord<dynamic>

using var reader = new StreamReader(newStream);
using var csvReader = new CsvReader(reader, config);
while (csvReader.Read())
{
    var stopwatch = Stopwatch.StartNew();
    var record = csvReader.GetRecord<dynamic>();
    logger.DebugLog(logParams, stopwatch.ElapsedMilliseconds, "done reading row");
    yield return record;
}
JoshClose commented 4 months ago

Dynamic is very slow. I can probably do some work on it to speed it up.