marscher
commented
6 years ago we can disable the progress bars globally for the testing session (eg. have a look at the conf_test.py of pyemma). From my experience I can tell, that bars are not causing too much overhead (tqmd costs like 800 ns per iteration + the overhead of drawing widgets). One concern about your benchmark is, that you can't really compare two runs on CI, because they are being executed on different hardware every time (+ different workloads on these machines). You can compare these runtimes locally to get an idea.
cwehmeyer
@marscher: it turns out that creating many short-lived progress bars is a huge slowdown and not very informative. Cells 5 and 4 (both do cross-validated vamp score calculations) are among the slowest parts in their container:
slowest cells with progress bars
``` =============================== Notebook timings =============================== ========================== slowest 30 test durations =========================== 97.76s call pyemma_tutorials/notebooks/00-pentapeptide-showcase.ipynb::Cell 5 79.44s call pyemma_tutorials/notebooks/07-expectations-and-observables.ipynb::Cell 11 66.77s call pyemma_tutorials/notebooks/07-expectations-and-observables.ipynb::Cell 12 54.61s call pyemma_tutorials/notebooks/00-pentapeptide-showcase.ipynb::Cell 1 42.59s call pyemma_tutorials/notebooks/00-pentapeptide-showcase.ipynb::Cell 4 ```Deactivating the progress bars locally speeds up these cells a lot on my machine and also on circleci:
slowest cells without progress bars
``` =============================== Notebook timings =============================== ========================== slowest 30 test durations =========================== 74.00s call pyemma_tutorials/notebooks/07-expectations-and-observables.ipynb::Cell 11 66.04s call pyemma_tutorials/notebooks/00-pentapeptide-showcase.ipynb::Cell 1 64.01s call pyemma_tutorials/notebooks/07-expectations-and-observables.ipynb::Cell 12 61.06s call pyemma_tutorials/notebooks/00-pentapeptide-showcase.ipynb::Cell 15 53.43s call pyemma_tutorials/notebooks/00-pentapeptide-showcase.ipynb::Cell 13 47.77s call pyemma_tutorials/notebooks/00-pentapeptide-showcase.ipynb::Cell 9 28.11s call pyemma_tutorials/notebooks/00-pentapeptide-showcase.ipynb::Cell 5 27.02s call pyemma_tutorials/notebooks/07-expectations-and-observables.ipynb::Cell 1 20.99s call pyemma_tutorials/notebooks/00-pentapeptide-showcase.ipynb::Cell 4 ```To this aim, I wrote a short context manager which might also be a good fit for pyemma:
no_prpgress_bars
```Python def no_progress_bars(): """Context manager to suppress progress bars. Example ------- Use inside a `with` statement: >>> with no_progress_bars(): >>> # operation which would print a progress bar """ class NoProgressBarsContextManager: def __init__(self): self.state = pyemma.config.show_progress_bars def __enter__(self): pyemma.config.show_progress_bars = False def __exit__(self, type, value, traceback): pyemma.config.show_progress_bars = self.state return NoProgressBarsContextManager() ```What is your take on this?