vincentlaucsb / csv-parser

A high-performance, fully-featured CSV parser and serializer for modern C++.
MIT License
864 stars 144 forks source link

CSVReader fails with files bigger than 10 MB when constructed with std::ifstream #202

Closed lamorlette-cintoo closed 1 month ago

lamorlette-cintoo commented 2 years ago

I used to construct CVSReader with a std::ifstream as follows:

const std::filesystem::path file_path {"foo"};
std::ifstream input(file_path , std::ios::binary | std::ios::in);
csv::CSVReader path_reader {input};

It is working perfectly fine with relatively small files, but it throws an exception with files bigger than 10 MB.

I found out that the same files work perfectly fine when using the other CVSReader constructor, taking the filename as parameter:

const std::filesystem::path file_path {"foo"};
csv::CSVReader path_reader {file_path.string()};

(Unfortunately I can't share with you an example because I don't own them.)

sjoubert commented 1 year ago

Duplicate of #180

vincentlaucsb commented 1 month ago

Closing as duplicate of #180. While this is an issue, I am curious as to why people are simply not using the default memory-mapped CSVReader as recommended in the README.