Closed lps-rocks closed 3 years ago
I have an early prototype for this. It wouldn't actually require changes to Enlighten, though it could be integrated to make it easier.
Essentially, this can be done in formatting by passing count
and total
with a type that supports formatting in this way. I looked at several libraries for this, but they all seemed to be missing at least one required feature, so I started writing one, Prefixed. Basically it provides an implementation of the built-in float that adds additional types to the format specification.
'h' for SI prefixes (..., n, μ, m, k, M, G, ...),
'j' for IEC prefixes (Ki, Mi, Gi, ...)
'J' for IEC prefixes without the i (K, M, G, ...).
It still needs a lot of work, but an early version is on PyPI and here's an example in use with Enlighten. Please try it out and let me know your thoughts.
import time
import random
import enlighten
from prefixed import Float
# 64k chunk size
chunk_size = 64 * 1024
# 1 - 10 MB file size
total = Float(random.randint(1, 10) * 2**20)
bar_format = '{desc}{desc_pad}{percentage:3.0f}%|{bar}| {count:.2j}B/{total:.2j}B ' \
'[{elapsed}<{eta}, {rate:.2j}{unit}/s]'
manager = enlighten.get_manager()
download = manager.counter(total=total, count=Float(), desc='Downloading',
unit='B', bar_format=bar_format)
bytes_left = total
while bytes_left:
time.sleep(random.uniform(0.05, 0.15))
next_chunk = min(chunk_size, bytes_left)
download.update(next_chunk)
bytes_left -= next_chunk
manager.stop()
Any updates on this? I can't make it work with prefixed because rate
is a float and it does not respect the j
format.
Oh, I think that changed because there is an explicit conversion to float in the math so prefixed.Float doesn't carry through to rate anymore.
I think I have a way for this to work. I had been thinking I'd covert the types that get stored and was stuck on how to manage that without too much disruption, but now I realize I only need to convert the fields that are stored for formatting. I really wish I had made these all floats, but total and count can be integers or floats, so I'll need a little logic there and Prefixed will need an Int class.
A quick look and I think these are the fields that need converting:
I'm not sure how soon I can get to this, but it's definitely on my radar.
I decided to skip integers and only implement it for floats. I'll probably force integers to floats in 2.0 (whenever that happens) anyway.
The change is in master. To enable, all you need to do is set your total
or count
to a float and everything else is automatic.
Please try it out and let me know what you think. An example is here. If you don't like having a space in 2.00 MiB
, just remove the !
in the format spec.
I updated the documentation here. It's mostly updated types and a note near the bottom of the section. There is probably a way to make it clearer, but I struggled with how best to provide the information. I think it still needs a dedicated section in Common Patterns.
Great, I like how it looks without the spaces. The only weird thing is the float casting, I hope 2.0 comes soon. Thank you!!
@jorgeecardona, can you explain more what you mean by "The only weird thing is the float casting". Just want to make the experience as smooth as I can.
Hi, I refer to the fact that I have to force total
to be a float (as you
clearly explained). For file processing, where I guess is very usual to
just deal with the size of the file, it looks strange (unpythonic maybe) to
have something like float(file.size)
. I understand why is needed at the
moment; I hope that changes once 2.0 is around.
On Mon, Mar 15, 2021 at 7:50 PM Avram Lubkin @.***> wrote:
@jorgeecardona https://github.com/jorgeecardona, can you explain more what you mean by "The only weird thing is the float casting". Just want to make the experience as smooth as I can.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Rockhopper-Technologies/enlighten/issues/29#issuecomment-799667375, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAYSXVWI2IXHW45C3TXA33TDZJIBANCNFSM4PXBATYA .
-- Jorge Eduardo Cardona
@. @.
cardona.co github.com/jorgeecardona
Ok. Yeah, no way around that now. People probably won't like using float formatting for integers in 2.0 either, but I'm trying to strike a balance. 2.0 will probably wait until I have all my backlog ideas done. I don't think any of the others are breaking changes, but I want to make sure.
Maybe in this direction it would be nice to write "seconds per iteration" instead of the rate when the rate is lower than 1, that seems to me more human readable.
I think you're looking for {interval}
(seconds per iteration). It's the inverse of {rate}
(iterations per second).
Is your feature request related to a problem? Please describe. For handling data, making the units able to be auto formatted as E/P/T/G/M/K{unit}ps would be awesome and improve human readability
I'm not sure if this is best achieved by code that changes the units on the fly, but being able to mix units would also be reuqired, e.g. if using bytes, MB/s for speed and K/G/T/PM/s for size / total transfered - also being able to specify how many decimal places would be useful when formatting. For example 3.22GB/10.23TB 170MB/s, is there a way to easily do this or a way to add custom fields that use a callback and the callback gets passed the current data to be formatted?
I thought about using custom fields but that would mean a lot of additional messy work.
thoughts?