Rockhopper-Technologies / enlighten

Enlighten Progress Bar for Python Console Apps
https://python-enlighten.readthedocs.io
Mozilla Public License 2.0
416 stars 25 forks source link

resize handler crashes when window resized and no counters are showing #64

Closed breathe closed 11 months ago

breathe commented 11 months ago

Describe the bug When the manager still exists but the enlighten.Counter()'s have all been .close(clear=True) there's a crash that can happen inside resize handler

│ .venv/lib/python3.10/site-packages/enlighten/_manager.py:113 in │
│ _stage_resize                                                                                    │
│                                                                                                  │
│   110 │   │                                                                                      │
│   111 │   │   else:                                                                              │
│   112 │   │   │   # If not threaded, handle resize now                                           │
│ ❱ 113 │   │   │   self._resize_handler()                                                         │
│   114 │                                                                                          │
│   115 │   def _resize_handler(self):                                                             │
│   116 │   │   """                                                                                │
│                                                                                                  │
│ .venv/lib/python3.10/site-packages/enlighten/_manager.py:135 in │
│ _resize_handler                                                                                  │
│                                                                                                  │
│   132 │   │                                                                                      │
│   133 │   │   if newHeight < oldHeight:                                                          │
│   134 │   │   │   buffer.append(term.move(max(0, newHeight - self.scroll_offset), 0))            │
│ ❱ 135 │   │   │   buffer.append(u'\n' * (2 * max(self.counters.values())))                       │
│   136 │   │   elif newHeight > oldHeight and self.threaded:                                      │
│   137 │   │   │   buffer.append(term.move(newHeight, 0))                                         │
│   138 │   │   │   buffer.append(u'\n' * (self.scroll_offset - 1))                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: max() arg is an empty sequence

To Reproduce

~I tried to extract a minimal repro but wasn't able to ... Logically the crash above happens where the print("crash on resize possible from here in our codebase...") message is printed ...~

Found a way to repro -- code updated below

def sleep_and_print():
    import time

    time.sleep(1)
    print("text")
    import warnings

    warnings.warn("test warning")

if __name__ == "__main__":
    import enlighten, time

    total = 3
    desc = "test"
    unit = "step"

    enlighten_manager = enlighten.get_manager(
        # tried to repro with variations of these -- but can't
        # no_resize=False,
        # threaded=False,
    )

    pbar = enlighten_manager.counter(
        total=total,
        desc=desc,
        unit=unit,
        leave=False,
    )

    import concurrent.futures

    with concurrent.futures.ThreadPoolExecutor() as executor:
        # the work we do while counter exists can either launch threads or not -- added some threads here to try and induce the error
        for i in range(total):
            future = executor.submit(time.sleep, 1)
            future.result()
        pbar.update()

    print("closing")
    pbar.close(clear=True)

    print(
        "crash on resize seems to happen from here in our codebase, after closing the counter but enlighten_manager still exists ..."
    )

    with concurrent.futures.ProcessPoolExecutor() as executor:
        while True:
            future = executor.submit(sleep_and_print)
            future.result()

Failure output below

Observations

(zephyrus-py3.10) ncohen@m1-max-toast ~/s/z/zephyrus (ben/fix-progressbar-error) [1]> python repro.py
closing
crash on resize seems to happen from here in our codebase, after closing the counter but enlighten_manager still exists ...
text
/Users/ncohen/software/zephr/zephyrus/repro.py:8: UserWarning: test warning
  warnings.warn("test warning")
text
text
text
text
text
text
text
text
text
text
text
text
Traceback (most recent call last):
  File "/Users/ncohen/software/zephr/zephyrus/repro.py", line 50, in <module>
    future.result()
  File "/Users/ncohen/.pyenv/versions/3.10.10/lib/python3.10/concurrent/futures/_base.py", line 453, in result
    self._condition.wait(timeout)
  File "/Users/ncohen/.pyenv/versions/3.10.10/lib/python3.10/threading.py", line 320, in wait
    waiter.acquire()
  File "/Users/ncohen/Library/Caches/pypoetry/virtualenvs/zephyrus--PefSXhl-py3.10/lib/python3.10/site-packages/enlighten/_manager.py", line 113, in _stage_resize
    self._resize_handler()
  File "/Users/ncohen/Library/Caches/pypoetry/virtualenvs/zephyrus--PefSXhl-py3.10/lib/python3.10/site-packages/enlighten/_manager.py", line 135, in _resize_handler
    buffer.append(u'\n' * (2 * max(self.counters.values())))
ValueError: max() arg is an empty sequence

Environment (please complete the following information):

Additional context

~Sorry I tried to simulate what happens in our environment to repro ... What is above is my attempt to extract a repro ...~

Repro above works for me -- for additional context

The full stack trace within our application is actually within matplotlib if that provides any clues ...

(full stack trace minus our application code which calls into pandas plotting) ... Maybe the code in matplotlib is printing something?

│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_core.py:100 │
│ 0 in __call__                                                                                    │
│                                                                                                  │
│    997 │   │   │   │   │   label_name = label_kw or data.columns                                 │
│    998 │   │   │   │   │   data.columns = label_name                                             │
│    999 │   │                                                                                     │
│ ❱ 1000 │   │   return plot_backend.plot(data, kind=kind, **kwargs)                               │
│   1001 │                                                                                         │
│   1002 │   __call__.__doc__ = __doc__                                                            │
│   1003                                                                                           │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_matplotlib/ │
│ __init__.py:71 in plot                                                                           │
│                                                                                                  │
│   68 │   │   │   │   ax = plt.gca()                                                              │
│   69 │   │   │   kwargs["ax"] = getattr(ax, "left_ax", ax)                                       │
│   70 │   plot_obj = PLOT_CLASSES[kind](data, **kwargs)                                           │
│ ❱ 71 │   plot_obj.generate()                                                                     │
│   72 │   plot_obj.draw()                                                                         │
│   73 │   return plot_obj.result                                                                  │
│   74                                                                                             │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_matplotlib/ │
│ core.py:458 in generate                                                                          │
│                                                                                                  │
│    455 │   │   self._adorn_subplots()                                                            │
│    456 │   │                                                                                     │
│    457 │   │   for ax in self.axes:                                                              │
│ ❱  458 │   │   │   self._post_plot_logic_common(ax, self.data)                                   │
│    459 │   │   │   self._post_plot_logic(ax, self.data)                                          │
│    460 │                                                                                         │
│    461 │   def _args_adjust(self):                                                               │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_matplotlib/ │
│ core.py:655 in _post_plot_logic_common                                                           │
│                                                                                                  │
│    652 │   def _post_plot_logic_common(self, ax, data):                                          │
│    653 │   │   """Common post process for each axes"""                                           │
│    654 │   │   if self.orientation == "vertical" or self.orientation is None:                    │
│ ❱  655 │   │   │   self._apply_axis_properties(ax.xaxis, rot=self.rot, fontsize=self.fontsize)   │
│    656 │   │   │   self._apply_axis_properties(ax.yaxis, fontsize=self.fontsize)                 │
│    657 │   │   │                                                                                 │
│    658 │   │   │   if hasattr(ax, "right_ax"):                                                   │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_matplotlib/ │
│ core.py:744 in _apply_axis_properties                                                            │
│                                                                                                  │
│    741 │   │   """                                                                               │
│    742 │   │   if rot is not None or fontsize is not None:                                       │
│    743 │   │   │   # rot=0 is a valid setting, hence the explicit None check                     │
│ ❱  744 │   │   │   labels = axis.get_majorticklabels() + axis.get_minorticklabels()              │
│    745 │   │   │   for label in labels:                                                          │
│    746 │   │   │   │   if rot is not None:                                                       │
│    747 │   │   │   │   │   label.set_rotation(rot)                                               │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:1413 in   │
│ get_majorticklabels                                                                              │
│                                                                                                  │
│   1410 │                                                                                         │
│   1411 │   def get_majorticklabels(self):                                                        │
│   1412 │   │   """Return this Axis' major tick labels, as a list of `~.text.Text`."""            │
│ ❱ 1413 │   │   self._update_ticks()                                                              │
│   1414 │   │   ticks = self.get_major_ticks()                                                    │
│   1415 │   │   labels1 = [tick.label1 for tick in ticks if tick.label1.get_visible()]            │
│   1416 │   │   labels2 = [tick.label2 for tick in ticks if tick.label2.get_visible()]            │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:1264 in   │
│ _update_ticks                                                                                    │
│                                                                                                  │
│   1261 │   │   """                                                                               │
│   1262 │   │   major_locs = self.get_majorticklocs()                                             │
│   1263 │   │   major_labels = self.major.formatter.format_ticks(major_locs)                      │
│ ❱ 1264 │   │   major_ticks = self.get_major_ticks(len(major_locs))                               │
│   1265 │   │   self.major.formatter.set_locs(major_locs)                                         │
│   1266 │   │   for tick, loc, label in zip(major_ticks, major_locs, major_labels):               │
│   1267 │   │   │   tick.update_position(loc)                                                     │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:1602 in   │
│ get_major_ticks                                                                                  │
│                                                                                                  │
│   1599 │   │                                                                                     │
│   1600 │   │   while len(self.majorTicks) < numticks:                                            │
│   1601 │   │   │   # Update the new tick label properties from the old.                          │
│ ❱ 1602 │   │   │   tick = self._get_tick(major=True)                                             │
│   1603 │   │   │   self.majorTicks.append(tick)                                                  │
│   1604 │   │   │   self._copy_tick_props(self.majorTicks[0], tick)                               │
│   1605                                                                                           │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:1551 in   │
│ _get_tick                                                                                        │
│                                                                                                  │
│   1548 │   │   │   │   f"The Axis subclass {self.__class__.__name__} must define "               │
│   1549 │   │   │   │   "_tick_class or reimplement _get_tick()")                                 │
│   1550 │   │   tick_kw = self._major_tick_kw if major else self._minor_tick_kw                   │
│ ❱ 1551 │   │   return self._tick_class(self.axes, 0, major=major, **tick_kw)                     │
│   1552 │                                                                                         │
│   1553 │   def _get_tick_label_size(self, axis_name):                                            │
│   1554 │   │   """                                                                               │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:417 in    │
│ __init__                                                                                         │
│                                                                                                  │
│    414 │   __name__ = 'xtick'                                                                    │
│    415 │                                                                                         │
│    416 │   def __init__(self, *args, **kwargs):                                                  │
│ ❱  417 │   │   super().__init__(*args, **kwargs)                                                 │
│    418 │   │   # x in data coords, y in axes coords                                              │
│    419 │   │   ax = self.axes                                                                    │
│    420 │   │   self.tick1line.set(                                                               │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:156 in    │
│ __init__                                                                                         │
│                                                                                                  │
│    153 │   │   │   grid_alpha = mpl.rcParams["grid.alpha"]                                       │
│    154 │   │   grid_kw = {k[5:]: v for k, v in kwargs.items()}                                   │
│    155 │   │                                                                                     │
│ ❱  156 │   │   self.tick1line = mlines.Line2D(                                                   │
│    157 │   │   │   [], [],                                                                       │
│    158 │   │   │   color=color, linestyle="none", zorder=zorder, visible=tick1On,                │
│    159 │   │   │   markeredgecolor=color, markersize=size, markeredgewidth=width,                │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/_api/deprecation. │
│ py:454 in wrapper                                                                                │
│                                                                                                  │
│   451 │   │   │   │   "positionally is deprecated since Matplotlib %(since)s; the "              │
│   452 │   │   │   │   "parameter will become keyword-only %(removal)s.",                         │
│   453 │   │   │   │   name=name, obj_type=f"parameter of {func.__name__}()")                     │
│ ❱ 454 │   │   return func(*args, **kwargs)                                                       │
│   455 │                                                                                          │
│   456 │   # Don't modify *func*'s signature, as boilerplate.py needs it.                         │
│   457 │   wrapper.__signature__ = signature.replace(parameters=[                                 │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/lines.py:381 in   │
│ __init__                                                                                         │
│                                                                                                  │
│    378 │   │                                                                                     │
│    379 │   │   self.set_markevery(markevery)                                                     │
│    380 │   │   self.set_antialiased(antialiased)                                                 │
│ ❱  381 │   │   self.set_markersize(markersize)                                                   │
│    382 │   │                                                                                     │
│    383 │   │   self._markeredgecolor = None                                                      │
│    384 │   │   self._markeredgewidth = None                                                      │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/lines.py:1254 in  │
│ set_markersize                                                                                   │
│                                                                                                  │
│   1251 │   │   │   self.stale = True                                                             │
│   1252 │   │   self._markeredgewidth = ew                                                        │
│   1253 │                                                                                         │
│ ❱ 1254 │   def set_markersize(self, sz):                                                         │
│   1255 │   │   """                                                                               │
│   1256 │   │   Set the marker size in points.                                                    │
│   1257                                                                                           │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/enlighten/_manager.py:113 in │
│ _stage_resize                                                                                    │
│                                                                                                  │
│   110 │   │                                                                                      │
│   111 │   │   else:                                                                              │
│   112 │   │   │   # If not threaded, handle resize now                                           │
│ ❱ 113 │   │   │   self._resize_handler()                                                         │
│   114 │                                                                                          │
│   115 │   def _resize_handler(self):                                                             │
│   116 │   │   """                                                                                │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/enlighten/_manager.py:135 in │
│ _resize_handler                                                                                  │
│                                                                                                  │
│   132 │   │                                                                                      │
│   133 │   │   if newHeight < oldHeight:                                                          │
│   134 │   │   │   buffer.append(term.move(max(0, newHeight - self.scroll_offset), 0))            │
│ ❱ 135 │   │   │   buffer.append(u'\n' * (2 * max(self.counters.values())))                       │
│   136 │   │   elif newHeight > oldHeight and self.threaded:                                      │
│   137 │   │   │   buffer.append(term.move(newHeight, 0))                                         │
│   138 │   │   │   buffer.append(u'\n' * (self.scroll_offset - 1))                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: max() arg is an empty sequence
avylove commented 11 months ago

Good catch! Thanks for reporting!

I can reproduce with:

import signal
import sys
import time

import enlighten

# Create a counter and close it
manager = enlighten.get_manager()
with manager.counter(leave=False) as pbar:
    pass

# Resize window and trigger signal
sys.stdout.write(f'\033[8;{manager.term.height - 2};{manager.term.width}t')
signal.raise_signal(signal.SIGWINCH)

# Wait for error
time.sleep(3)

I think it's an easy fix, but will need to come up with test code. I'll try to have a new release out soon.

avylove commented 11 months ago

This should be fixed in 1.12.4. Please try it out and let me know. Thanks!