Sub-counters not available in counter_format

[edmcman]

I would really like to display some other statistics/counters when in counter mode. It seems like the only way to do this is using sub-counters, but if I try to access a sub-counter in a counter_format, I get an error such as:

KeyError: 'count_1'

[avylove]

I think it would help to see some of the code. Can you post what you're doing or some sample code that creates the condition?

[edmcman]

#!/usr/bin/python

import enlighten

# Setup progress bar
manager = enlighten.get_manager()
bar_format = u'{desc}{desc_pad}{count:d} {unit}{unit_pad}{elapsed} ' + \
             u'S:{count_1} ' + \
             u'[{elapsed}<{eta}, {rate:.2f}{unit_pad}{unit}/s]'
with manager.counter(desc='Steps', unit='steps', counter_format=bar_format) as pbar:

    goalctr = pbar.add_subcounter(42)

    for x in range(100):
        pbar.update()
        goalctr.update()

[edmcman]

More generally though I just want to output some statistics. For instance, if I could say pbar.update(my_var1=10, my_var2=20) and reference those in the format, it would solve my use case too.

[avylove]

count_n only works with bar_format, not counter_format. I guess I could make it work with both. I guess I hadn't thought of this use case. Subcounters were designed for multicolored progress bars, where one color represents a subset of the total count. I didn't think of it terms of showing sub-statistics when no total was given.

Basically the method that add the fields for the subcounters never gets called when there is no total or the count is higher than the total. I can add that, but probably not in the next week or two.

A workaround for now would be to use bar_format and set the total to the final number, if you know it, or a number you know you won't hit. Keep in mind, the eta field is based on total, so if the number is not known, eta doesn't really add value.

[edmcman]

Yes, that is exactly the bug. My problem is unbounded so I am using counter_format. What do you think of passing data through update args?

[avylove]

That seems like something that would be useful. Basically extra fields that get passed along to format(). Would just have to catch key errors and give a more useful exception like Keyword 'my_var2' specified, but not provided'

[edmcman]

Right. Do you want me to take a first pass on implementing this?

[avylove]

You're welcome to. I won't have a chance to look at it for a week or two, but I have some other changes I need to incorporate so I'll try to get everything in for a new release.

[avylove]

I had a chance to look into it today. I'll be able to add arbitrary fields, but it's going to work a little differently than you were thinking. Something like this, where you'd specify a dictionary of additional fields when instantiating a manager or counter. The contents of the dictionary will will loaded every time format() is called, so the contents can be changed as long as the key specified in the counter or bar format isn't removed.

import enlighten
import time

manager = enlighten.get_manager()
counter_format = u'{desc}{desc_pad}{count:d} {unit}{unit_pad}' + \
                 u'S:{my_var1} ' + \
                 u'[{elapsed}, {rate:.2f}{unit_pad}{unit}/s]{fill}'
my_fields = {'my_var1': 42}
with manager.counter(desc='Steps', unit='steps', counter_format=counter_format,
                     additional_fields=my_fields) as pbar:
    for x in range(100):
        time.sleep(0.05)
        my_fields['my_var1'] += 1
        pbar.update()

Do you think that will work?

[edmcman]

That would be great! Obviously I didn't get a chance to work on it myself 🙂

On Mon, Feb 17, 2020, 10:24 PM Avram Lubkin - notifications@github.com github.edmcman.99c9f1b9d0.notifications#reply@reply.github.com wrote:

I had a chance to look into it today. I'll be able to add arbitrary fields, but it's going to work a little differently than you were thinking. Something like this, where you'd specify a dictionary of additional fields when instantiating a manager or counter. The contents of the dictionary will will loaded every time format() is called, so the contents can be changed as long as the key specified in the counter or bar format isn't removed.

import enlightenimport time

manager = enlighten.get_manager() counter_format = u'{desc}{desc_pad}{count:d} {unit}{unit_pad}' + \ u'S:{my_var1} ' + \ u'[{elapsed}, {rate:.2f}{unit_pad}{unit}/s]{fill}' my_fields = {'my_var1': 42}with manager.counter(desc='Steps', unit='steps', counter_format=counter_format, additional_fields=my_fields) as pbar: for x in range(100): time.sleep(0.05) my_fields['my_var1'] += 1 pbar.update()

Do you think that will work?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Rockhopper-Technologies/enlighten/issues/12?email_source=notifications&email_token=AAHYKZOMO2NO62JES2JUE6DRDNIF5A5CNFSM4KPHBTKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMAOFHA#issuecomment-587260572, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHYKZMZFBIOGFJ5OX2Z4JTRDNIF5ANCNFSM4KPHBTKA .

[avylove]

Just added aa2b6f783d0337d562688abc554a22d45c85062b. Can you test it out?

[edmcman]

It works great. Thanks!

[avylove]

Both additional_fields and subcounter fields for counter format were released with 1.5.0

[avylove]

In the latest release, I left additional_fields, but added a new way to do this using keywords. It's described here. You may want to also look at the new status bar feature.

[edmcman]

Thanks for the update! I'll migrate to using keywords in case you decide to deprecate additional_fields