search

ben-strasser / fast-cpp-csv-parser

fast-cpp-csv-parser
BSD 3-Clause "New" or "Revised" License
2.11k stars 440 forks source link

Multiple input files #74

Closed ncoder-1 closed 5 years ago

ncoder-1 commented 5 years ago

Hi,

I was wondering if there's a built-in way to open multiple (same layout) csv files into one CSVReader object? I have 2 (sometimes 3) csv files that have the exact same column structure that I would like to open into one concatenated CSVReader object.

I am trying to avoid concatenating the csv files on disk prior to reading them with CSVReader and I'd like to avoid having 2-3 individual CSVReader objects to handle.

Basically, I'm wondering if there's another input option (say a vector of strings) instead of a direct filename such as below:

io::CSVReader<8, io::trim_chars<' '>, io::double_quote_escape<',', '\"'> > in(Filename);

Thanks!

ben-strasser commented 5 years ago

Hi,

there is no built in way.

What I have personally done in the past to achieve this was to implement was to implement my own ByteSource. The derived class essentially contained an ifstream, a counter, and a basename. The class first tried to open basename + to_string(counter). If it succeeded, it read the content, incremented the counter, and then tried to open basename + to_string(counter). If opening failed, it returned 0 to indicate an EOF.

(The class also additionally implemented on-the-fly zip decompression but that is beside the point of the question.)

Best Regards Dr. Ben Strasser

On 09/27/2018 05:56 PM, elimpnick wrote:

Hi,

I was wondering if there's a built-in way to open multiple (same layout) csv files into one CSVReader object? I have 2 (sometimes 3) csv files that have the exact same column structure that I would like to open into one concatenated CSVReader object.

I am trying to avoid concatenating the csv files on disk prior to reading them with CSVReader and I'd like to avoid having 2-3 individual CSVReader objects to handle.

Basically, I'm wondering if there's another input option instead of a direct filename such as below:

|io::CSVReader<8, io::trim_chars<' '>, io::double_quote_escape<',', '\"'> > in(Filename);|

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ben-strasser/fast-cpp-csv-parser/issues/74, or mute the thread https://github.com/notifications/unsubscribe-auth/ALaAj5U2z-w7xSsjyWWsEdBiEIsB6T_Kks5ufPUVgaJpZM4W897k.

ncoder-1 commented 5 years ago

I see, thanks for the answer and the great library!