UK-IPOP / drug-extraction

A ToolBox for fuzzily extracting drugs mentions from text.
https://drug-extraction.vercel.app
MIT License
3 stars 0 forks source link

Progress bar shows invalid progress when parsing multi-line csv records #78

Closed nanthony007 closed 2 years ago

nanthony007 commented 2 years ago

Self explanatory title

Would like to show each "record" iteration as opposed to iterating over lines.

Imagine a csv with large cells of text that take up many lines and contain newlines, we don't want to show a progress over these lines but over the records.

Right now the numerator of our progress bar iterates correctly because we manually increment. Thus I think to fix this we simply need to adjust the denominator line counting/file-scanning function. Which is the following line:

let line_count = BufReader::new(File::open(&ssi.fpath)?).lines().count();