search

taraslayshchuk / es2csv

Export from an Elasticsearch into a CSV file
Apache License 2.0
511 stars 190 forks source link

better progressbar unit names #2

Closed pokab closed 8 years ago

pokab commented 8 years ago

The default "byte" was misleading.

taraslayshchuk commented 8 years ago

kdocs per second and klines per second look like not clear for me.

pokab commented 8 years ago

OK. Are we on the same page with respect to bytes and kB being totally wrong here? As I understand Elasticsearch concepts, documents are being fetched here, so that is why I thought "doc" was ok. Maybe "Documents" would be better. When converting to CSV, lines are being processed, so I wrote "lines". What would you like?

taraslayshchuk commented 8 years ago

I think kB/s has a place to be there, because it shows speed of downloading data and writing to disc. For documents and lines we have counter:

Run query [##################] [91440/91440] [100%] [0:00:07] [Time: 0:00:07] [ 12.62 kB/s]

                                /         \
                      current doc         total docs

Write to csv [########____] [63206/91440] [ 69%] [0:00:01] [ETA: 0:00:00] [ 31.61 kB/s]

                                /         \
                      current line         total lines

And anyway the measurements kdocs/s and klines/s confuses me, so it could be just removed.

pokab commented 8 years ago

I think kB/s has a place to be there, because it shows speed of downloading data and writing to disc.

The only reason for me to touch this was that it is not the speed of the download or writing to disk. Instead of bytes, the program accounts for downloaded documents (records) and lines instead. It has no information about bytes at all. The actual speed is orders of magnitude faster (as it should be), which is why I discovered this in the first place. I'm not sure why you're confused about non-byte/s measurement, it is simply the speed of the progressbar filling up.

taraslayshchuk commented 8 years ago

I reviewed scrutinized speed values which you talking about and I realized that you are right, it is really docs per second, just in my case it is kdocs(1024*docs) and in some case it is shock. Good job, thank you!