ben-strasser / fast-cpp-csv-parser

fast-cpp-csv-parser
BSD 3-Clause "New" or "Revised" License
2.09k stars 435 forks source link

Progressbar support #129

Closed KOLANICH closed 2 years ago

KOLANICH commented 2 years ago

I have implemented progressbar support (by just exposing the data from private fields) in my inlined version of your lib, but I'm not sure it matches to your vision of this lib.

ben-strasser commented 2 years ago

Can you do your progress bar as follows?

int main(){

  MyProgressBar progress_bar(number_of_lines);
  io::CSVReader<3> in("ram.csv");
  in.read_header(io::ignore_extra_column, "vendor", "size", "speed");
  std::string vendor; int size; double speed;
  while(in.read_row(vendor, size, speed)){
    progress_bar.update_progress(in.get_file_line());
    // do stuff with the data
  }
}

This way the whole progress bar logic is completely decoupled from the CSV parsing.

KOLANICH commented 2 years ago

Since we don't know the count of lines beforehand (in the case of non-indexed parsing), and since estimating by rows is incorrect, since rows are usually of different size, I took another approach:

  1. I measure the size of the file
  2. I have modified this lib to expose the positions of "cursors" within chunks
  3. From the positions of cursors I derive the count of bytes read
  4. Then the percentage is bytes read / file size.