turicas / rows

A common, beautiful interface to tabular data, no matter the format
GNU Lesser General Public License v3.0
865 stars 136 forks source link

[pgimport] Wrong imported bytes #370

Open turicas opened 1 year ago

turicas commented 1 year ago

The progress bar is not updated correctly: number of imported bytes is lower than file size (at least for uncompressed CSV files).

Giving the file test.csv (305 bytes):

Project,Client,Description,Duration
P23,Client ÁÁÁÀçç,Some task num. 01,01:08:56
Project N. 12,Client num. 1,Some task number 02,00:55:00
Proj 2,Client num. 1,Some task number 123456789012,06:31:17
Project 2,Client num. 1,Some task number 123456,01:18:49
Project 2,Client num. 1,Task n. 12,02:04:23

By running:

rows pgimport -s :text: -d excel -e utf-8 test.csv $DATABASE_URL test

I got the result:

5 rows imported:  88%|████████████████████████████████████████▌     | 269/305 [00:00<00:00, 6.28kbytes/s]

It seems the header size is not being counted. We may change the header, so we need to add to progress the original header size (currently even the written header size is not being added to imported bytes).

Note: the file is in UTF-8 and even if it haven't special chars, the imported bytes will still be less than file size.