Rockhopper-Technologies / enlighten

Enlighten Progress Bar for Python Console Apps
https://python-enlighten.readthedocs.io
Mozilla Public License 2.0
421 stars 25 forks source link

Human Readable Unit multipliers #29

Closed lps-rocks closed 3 years ago

lps-rocks commented 4 years ago

Is your feature request related to a problem? Please describe. For handling data, making the units able to be auto formatted as E/P/T/G/M/K{unit}ps would be awesome and improve human readability

I'm not sure if this is best achieved by code that changes the units on the fly, but being able to mix units would also be reuqired, e.g. if using bytes, MB/s for speed and K/G/T/PM/s for size / total transfered - also being able to specify how many decimal places would be useful when formatting. For example 3.22GB/10.23TB 170MB/s, is there a way to easily do this or a way to add custom fields that use a callback and the callback gets passed the current data to be formatted?

I thought about using custom fields but that would mean a lot of additional messy work.

thoughts?

avylove commented 4 years ago

I have an early prototype for this. It wouldn't actually require changes to Enlighten, though it could be integrated to make it easier.

Essentially, this can be done in formatting by passing count and total with a type that supports formatting in this way. I looked at several libraries for this, but they all seemed to be missing at least one required feature, so I started writing one, Prefixed. Basically it provides an implementation of the built-in float that adds additional types to the format specification. 'h' for SI prefixes (..., n, μ, m, k, M, G, ...), 'j' for IEC prefixes (Ki, Mi, Gi, ...) 'J' for IEC prefixes without the i (K, M, G, ...).

It still needs a lot of work, but an early version is on PyPI and here's an example in use with Enlighten. Please try it out and let me know your thoughts.

import time
import random

import enlighten
from prefixed import Float

# 64k chunk size
chunk_size = 64 * 1024

# 1 - 10 MB file size
total = Float(random.randint(1, 10) * 2**20)

bar_format = '{desc}{desc_pad}{percentage:3.0f}%|{bar}| {count:.2j}B/{total:.2j}B ' \
             '[{elapsed}<{eta}, {rate:.2j}{unit}/s]'

manager = enlighten.get_manager()
download = manager.counter(total=total, count=Float(), desc='Downloading',
                           unit='B', bar_format=bar_format)

bytes_left = total
while bytes_left:
    time.sleep(random.uniform(0.05, 0.15))
    next_chunk = min(chunk_size, bytes_left)
    download.update(next_chunk)
    bytes_left -= next_chunk

manager.stop()
jorgeecardona commented 3 years ago

Any updates on this? I can't make it work with prefixed because rate is a float and it does not respect the j format.

avylove commented 3 years ago

Oh, I think that changed because there is an explicit conversion to float in the math so prefixed.Float doesn't carry through to rate anymore.

I think I have a way for this to work. I had been thinking I'd covert the types that get stored and was stuck on how to manage that without too much disruption, but now I realize I only need to convert the fields that are stored for formatting. I really wish I had made these all floats, but total and count can be integers or floats, so I'll need a little logic there and Prefixed will need an Int class.

A quick look and I think these are the fields that need converting:

I'm not sure how soon I can get to this, but it's definitely on my radar.

avylove commented 3 years ago

I decided to skip integers and only implement it for floats. I'll probably force integers to floats in 2.0 (whenever that happens) anyway.

The change is in master. To enable, all you need to do is set your total or count to a float and everything else is automatic.

Please try it out and let me know what you think. An example is here. If you don't like having a space in 2.00 MiB, just remove the ! in the format spec.

I updated the documentation here. It's mostly updated types and a note near the bottom of the section. There is probably a way to make it clearer, but I struggled with how best to provide the information. I think it still needs a dedicated section in Common Patterns.

jorgeecardona commented 3 years ago

Great, I like how it looks without the spaces. The only weird thing is the float casting, I hope 2.0 comes soon. Thank you!!

avylove commented 3 years ago

@jorgeecardona, can you explain more what you mean by "The only weird thing is the float casting". Just want to make the experience as smooth as I can.

jorgeecardona commented 3 years ago

Hi, I refer to the fact that I have to force total to be a float (as you clearly explained). For file processing, where I guess is very usual to just deal with the size of the file, it looks strange (unpythonic maybe) to have something like float(file.size). I understand why is needed at the moment; I hope that changes once 2.0 is around.

On Mon, Mar 15, 2021 at 7:50 PM Avram Lubkin @.***> wrote:

@jorgeecardona https://github.com/jorgeecardona, can you explain more what you mean by "The only weird thing is the float casting". Just want to make the experience as smooth as I can.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Rockhopper-Technologies/enlighten/issues/29#issuecomment-799667375, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAYSXVWI2IXHW45C3TXA33TDZJIBANCNFSM4PXBATYA .

-- Jorge Eduardo Cardona

@. @.

cardona.co github.com/jorgeecardona

avylove commented 3 years ago

Ok. Yeah, no way around that now. People probably won't like using float formatting for integers in 2.0 either, but I'm trying to strike a balance. 2.0 will probably wait until I have all my backlog ideas done. I don't think any of the others are breaking changes, but I want to make sure.

jorgeecardona commented 3 years ago

Maybe in this direction it would be nice to write "seconds per iteration" instead of the rate when the rate is lower than 1, that seems to me more human readable.

avylove commented 3 years ago

I think you're looking for {interval} (seconds per iteration). It's the inverse of {rate} (iterations per second).

avylove commented 3 years ago

Released in 1.9.0.

Going to close this. Let me know how it works out.