search

simonw / csvs-to-sqlite

Convert CSV files into a SQLite database
Apache License 2.0
866 stars 71 forks source link

Progress bars would be useful #66

Open simonw opened 4 years ago

simonw commented 4 years ago

I'm processing a 100MB+ file at the moment and there's no indication of how long it will take.

simonw commented 4 years ago

This will be a lot easier to do without Pandas, see #69

simonw commented 4 years ago

I can use the same pattern for measuring progress as I do in healthkit-to-sqlite: use the number of bytes read from the underlying file as an indication of progress through it. https://github.com/dogsheep/healthkit-to-sqlite/blob/fe856bf313e269edec46d304fd09348f37282f1b/healthkit_to_sqlite/utils.py#L4-L19