Closed iced closed 6 years ago
happy to accept a PR, I didn't write the notebook bit
Seems it's ipywidgets issue (or even expected behavior).
definitely upstream, i have the same problem
Just wanted to confirm that I've experienced the same issue on Ubuntu 16.10 -- anyone have a hack to get around this? Is there a way to restart a progressbar?
I'm never sure what system/configuration information to provide, but here's my best guess:
$ conda --version
conda 4.5.1
$ conda list --name <redacted>
# packages in environment at <redacted>:
#
# Name Version Build Channel
alabaster 0.7.10 py36h306e16b_0
appdirs 1.4.3 <pip>
asn1crypto 0.24.0 py36_0
babel 2.5.3 py36_0
backcall 0.1.0 py36_0
bleach 2.1.3 py36_0
ca-certificates 2018.03.07 0
certifi 2018.1.18 py36_0
cffi 1.11.5 py36h9745a5d_0
chardet 3.0.4 py36h0f667ec_1
cryptography 2.2.2 py36h14c3975_0
cycler 0.10.0 py36h93f1223_0
dbus 1.13.2 h714fa37_1
decorator 4.3.0 py36_0
docutils 0.14 py36hb0f60f5_0
entrypoints 0.2.3 py36h1aec115_2
expat 2.2.5 he0dffb1_0
fontconfig 2.12.6 h49f89f6_0
freetype 2.8 hab7d2ae_1
ftperiodogram 1.0.0 <pip>
glib 2.56.1 h000015b_0
gmp 6.1.2 h6c8ec71_1
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
html5lib 1.0.1 py36h2f9c1c0_0
icu 58.2 h9c2bf20_1
idna 2.6 py36h82fb2a8_1
imagesize 1.0.0 py36_0
intel-openmp 2018.0.0 8
ipykernel 4.8.2 py36_0
ipython 6.3.1 py36_0
ipython_genutils 0.2.0 py36hb52b0d5_0
jedi 0.12.0 py36_0
jinja2 2.10 py36ha16c418_0
jpeg 9b h024ee3a_2
jsonschema 2.6.0 py36h006f8b5_0
jupyter_client 5.2.3 py36_0
jupyter_contrib_core 0.3.3 py36_1 conda-forge
jupyter_contrib_nbextensions 0.4.0 py36_0 conda-forge
jupyter_core 4.4.0 py36h7c827e3_0
jupyter_highlight_selected_word 0.1.0 py36_0 conda-forge
jupyter_latex_envs 1.4.4 py36_0 conda-forge
jupyter_nbextensions_configurator 0.4.0 py36_0 conda-forge
jupyterlab 0.31.12 py36_0
jupyterlab_launcher 0.10.5 py36_0
kiwisolver 1.0.1 py36h764f252_0
libedit 3.1 heed3624_0
libffi 3.2.1 hd88cf55_4
libgcc-ng 7.2.0 hdf63c60_3
libgfortran-ng 7.2.0 hdf63c60_3
libpng 1.6.34 hb9fc6fc_0
libsodium 1.0.16 h1bed415_0
libstdcxx-ng 7.2.0 hdf63c60_3
libxcb 1.13 h1bed415_1
libxml2 2.9.8 hf84eae3_0
libxslt 1.1.32 h1312cb7_0
lxml 4.2.1 py36h23eabaa_0
Mako 1.0.7 <pip>
markupsafe 1.0 py36hd9260cd_1
matplotlib 2.2.2 py36h0e671d2_1
mistune 0.8.3 py36_0
mkl 2018.0.2 1
mkl_fft 1.0.1 py36h3010b51_0
mkl_random 1.0.1 py36h629b387_0
more-itertools 4.1.0 <pip>
mwdust 1.0 <pip>
nbconvert 5.3.1 py36hb41ffb7_0
nbformat 4.4.0 py36h31c9010_0
ncurses 6.0 h9df7e31_2
nfft 0.1 <pip>
notebook 5.4.1 py36_0
numpy 1.14.2 <pip>
numpy 1.14.2 py36hdbf6ddf_1
numpydoc 0.8.0 py36_0
openssl 1.0.2o h20670df_0
packaging 17.1 py36_0
pandoc 1.19.2.1 hea2e7c5_1
pandocfilters 1.4.2 py36ha6701b7_1
parso 0.2.0 py36_0
pcre 8.42 h439df22_0
pexpect 4.5.0 py36_0
pickleshare 0.7.4 py36h63277f8_0
pip 9.0.2 <pip>
pip 9.0.3 py36_0
prompt_toolkit 1.0.15 py36h17d85b1_0
ptyprocess 0.5.2 py36h69acd42_0
py 1.5.3 <pip>
pycparser 2.18 py36hf9f622e_1
pycuda 2017.1.1 <pip>
pygments 2.2.0 py36h0d3125c_0
pyhmc 0.1.2 <pip>
pyopenssl 17.5.0 py36h20ba746_0
pyparsing 2.2.0 py36hee85983_1
pyqt 5.9.2 py36h751905a_0
pysocks 1.6.8 py36_0
pytest 3.5.0 <pip>
pytest 3.4.2 <pip>
python 3.6.5 hc3d631a_0
python-dateutil 2.7.2 py36_0
pytools 2018.3 <pip>
pytools 2018.2 <pip>
pytz 2018.4 py36_0
pyyaml 3.12 py36hafb9ca4_1
pyzmq 17.0.0 py36h14c3975_0
qt 5.9.5 h7e424d6_0
readline 7.0 ha6073c6_4
requests 2.18.4 py36he2e5f8d_1
scikit-cuda 0.5.1 <pip>
scikit-cuda 0.5.2 <pip>
send2trash 1.5.0 py36_0
setuptools 39.0.1 <pip>
setuptools 38.5.2 <pip>
setuptools 39.0.1 py36_0
simplegeneric 0.8.1 py36_2
sip 4.19.8 py36hf484d3e_0
six 1.11.0 py36h372c433_1
sklearn 0.0 <pip>
snowballstemmer 1.2.1 py36h6febd40_0
sphinx 1.7.2 py36_0
sphinxcontrib 1.0 py36h6d0f590_1
sphinxcontrib-websupport 1.0.1 py36hb5cb234_1
sqlite 3.22.0 h1bed415_0
terminado 0.8.1 py36_1
testpath 0.3.1 py36h8cadb63_0
tk 8.6.7 hc745277_3
tornado 5.0.1 py36_1
tqdm 4.19.7 <pip>
traitlets 4.3.2 py36h674d592_0
typing 3.6.4 py36_0
urllib3 1.22 py36hbe7ace6_0
wcwidth 0.1.7 py36hdf4376a_0
webencodings 0.5.1 py36h800622e_1
wheel 0.31.0 py36_0
xz 5.2.3 h55aa19d_2
yaml 0.1.7 had09818_2
zeromq 4.2.5 h439df22_0
zlib 1.2.11 ha838bed_2
This is (or rather was) an ipywidgets issue (#1845).
You can fix it by updating ipywidgets to version 7.2.0
@casperdcl since the upstream fixed the bug, this issue can be closed.
Thanks for your responses; I am still not able to fix this problem however, and I've got ipywidgets 7.2.1
and tqdm 4.19.7
. Should I be posting in jupyter-widgets/ipywidgets#1845 about this instead?
Below is my best attempt to reproduce the problem
#!/bin/sh
# verbose
set -x
PYTHON=3.6
ENV=problem
ANACONDA_DIR=~/anaconda3
# Not sure about what to delegate to pip/conda/conda-forge
CONDA_DEPS="pip"
PIP_DEPS="nodejs ipywidgets jupyter jupyterlab numpy tqdm"
CONDA_FORGE_DEPS="jupyter_contrib_nbextensions "
CONDA_FORGE_DEPS+="jupyter_nbextensions_configurator "
# whether we want to remove the conda environment and start anew
REINSTALL="yes"
# create a new conda environment to recreate the issue
function make_conda_env(){
if [[ $(conda env list | grep $ENV) == "" ]]; then
conda create --yes --name $ENV python=$PYTHON $CONDA_DEPS
fi
}
# clean up conda environment
function rm_conda_env(){
if [[ $(conda env list | grep $ENV) != "" ]]; then
conda remove --yes --name $ENV --all
fi
}
# an incorrect import statement in pip 10.0.0 makes it unusable
function fix_pip(){
pip_file=$ANACONDA_DIR/envs/$ENV/bin/pip
sed -i.bak 's/from pip/from pip._internal/g' $pip_file > $pip_file
}
# remove environment if its already present
if [[ $REINSTALL == "yes" ]]; then
rm_conda_env || exit "could not remove conda environment $ENV"
fi
# Set up conda environment
make_conda_env &&
source activate $ENV &&
# Install things
fix_pip &&
pip install --upgrade $PIP_DEPS &&
conda install --yes --channel conda-forge $CONDA_FORGE_DEPS &&
# update npm?
npm i -g npm &&
# Not necessary?
#jupyter serverextension enable --py jupyterlab --sys-prefix &&
#jupyter nbextension install --py --sys-prefix widgetsnbextension &&
# installing extensions?
jupyter nbextension enable --py --sys-prefix widgetsnbextension &&
jupyter contrib nbextension install --sys-prefix &&
jupyter labextension install @jupyter-widgets/jupyterlab-manager &&
# launch!
jupyter lab
import time
from tqdm import tqdm_notebook
n = 10
for i, x in tqdm_notebook(enumerate(range(n)), total=n):
for j, y in tqdm_notebook(enumerate(range(n)), total=n, leave=False):
time.sleep(0.1)
which reproduces the issue for me.
Also: import ipywidgets; print(ipywidgets.__version__)
(from within the notebook) gives me 7.2.1
@johnh2o2 eh, your code in 2.
works perfectly on my jupyter-notebook. I don't have jupyter lab, but can you try first update tqdm
to the newest version?
can you run this in a jupyter lab cell?
from ipywidgets import Layout, Button, Box
from IPython.display import display
items = [Button(description='example') ]
box1 = Box(children=items)
box2 = Box(children=items)
box3= Box(children=items)
display(box1)
display(box2)
display(box3)
box2.close()
If it works, you will see two buttons, between them, there is no redundant space. However, if you see a redundant space, it means that this is not because of tqdm
and you still need to update and solve the compatibility between jupyter and ipywidgets.
I'm still getting this issue in JupyterLab (version 0.32.0) with ipywidgets
version 7.2.1 and tqdm
version 4.23.4. I ran @chengs code above, and the boxes in my notebook don't seem to have any more space between them than the boxes in your picture here (see below; is a significant amount (e.g. the size of a button) of redundant space expected?). Does this imply that it is a tqdm
issue then?
Hi all,
Sorry for the delay. I've updated tqdm
(4.23.4), jupyter
(4.4.0), jupyter lab
(0.32.1) (still have ipywidgets
7.2.1) and am still having issues. @chengs I have run your code in a jupyter lab cell (as well as printing out version numbers and trying the tqdm_notebook
again); here is the screenshot:
I'm not sure if the space between the buttons is redundant or not but I can confirm tqdm_notebook
still produces the extra space between inner and outer progress bars. Let me know if I can run any more tests on my machine (or another machine). I'm on Ubuntu 16.04.4 LTS in case that matters.
Oops -- that screenshot was from before I updated. Here's what it looks like now...
@johnh2o2 The space you see between the buttons is indeed redundant. I way to check this is to comment out the 'display(box2)' and 'box2.close()' lines and compare the whitespace. In my case, the whitespace is lesser if I never display the box, as opposed to displaying and then closing it. Note that I have ipywidgets 7.2.1 and tqdm 4.23.4 as well. Looks like it is an upstream issue with jupyterlab and ipywidgets. ps The only solution for now seems to somehow put the dynamically created progressbars in a single VBox...
Are you using jupyterthemes? In jupyter notebooks get extra spaces after tqdm progress bars when I use jupyterthemes styling, but not when I use the default CSS (and if someone can let me know which CSS property in jupyterthemes to change to get around that I'd be very grateful!)
I can confirm that this issue, while being fixed in jupyter notebook, still persists in jupyterlab (0.33.x).
This might have something to do with jupyterlab-manager, the ipywidgets jupyterlab extension.
@psmaragdis I am using jupyterthemes and I have this behavior.
Shouldn't this issue stay open as long as it's not fixed in upstream? Still experiencing this issue, which makes nested bars unusable in most cases with Jupyter lab...
I've made a workaround for this issue, but later I've noticed that there is a prettier version that manipulates css.
Since I heavily rely on multiprocessing with tqdm, there was a bug that spammed a lot of warnings about trying to display something from a child process. However, it worked just fine if you do a print('\n', end='\r')
before initialising tqdm
. But it would fail to remove blank spaces if after that more processes spawn with tqdm
. So I've made a little js snippet that traverses the DOM and deletes all the elements that are leaving only newlines.
N.B.! it will do this across the entire notebook not only your active cell (for that you've to query for classes like cell
and selected
).
I have a very short history with js, so I'm sure it can be optimised, but also easily modified for your use-case.
from IPython import display
display.display_javascript(r'''
var list = document.querySelectorAll('.output_text > pre');
for (var i = 0; i < list.length; i++) {
list[i].textContent = list[i].textContent.replace(/(\r\n|\n|\r)/gm,"");
if(list[i].innerHTML==="") {
list[i].closest(".output_area").remove();
}
}''', raw=True)`
What is the status of this fix on Jupyter Lab? I use ipywidgets 7.5.1, jupyterlab 1.1.1, and tqdm 4.35.0 and the issue still persists.
based on comments this seems to be an inconsistently reproducible upstream issue. it doesn't seem to be tqdm
-specific so I'm leaving this closed for now (the alternative would be every repo which uses jupyterlab has an issue open).
Would be great if someone could cross-ref an uptream issue number here, though.
I wasn't able to find an issue upstream that would pertain to this problem. I may just have missed it. However, I created an issue here just to be sure. Reason being: problem still exists after a long time. Maybe they're not aware of it.
I have the same issue with space in jupyter-notebook. The issue only appears after tqdm has run once in the session. Afterwards, each additional time tqdm is run, one more empty row will appear between the status bars with each iteration having one bar. Both ipywidges and tqdm are up to date.
I have encountered the same issue and modified Nyanguy's workaround to hide leftovers of ipywidgets progress bars.
from IPython import display
def hide_leftover_progress_bars():
display.display_javascript(r'''
var list = document.querySelectorAll('div.lm-Widget.jp-OutputArea-child > div.jupyter-widgets.jp-OutputArea-output.lm-mod-hidden');
for (var i = 0; i < list.length; i++) {
list[i].parentNode.classList.add("lm-mod-hidden");
}''', raw=True)
Removing the leftover container nodes from cell's output, rather than hiding them, will cause a crash when the cell is re-ran and widgets.js attempts to clean up these already gone nodes.
I returned to this issue after several years and was disappointed to learn that it had not yet been fixed.
I'm using tqdm 4.51.0 and ipywidgets 7.5.1 in jupyterlab 2.2.9.
Could the workarounds described above somehow be incorporated into tqdm or ipywidgets?
I am experiencing this issue too.
This is (or rather was) an ipywidgets issue (#1845).
You can fix it by updating ipywidgets to version 7.2.0
still experiencing with ipywidgets 7.6.3
Is there a possibility of reopenning this issue?
Just a little psychological. The issue seems to be persisting, and would be good to see acknowledgement to know we aren't going crazy
I would also like to see this issue reopened. Seeing the same issues described in the latest version of VSCode Insiders as well:
Just to re-confirm the real problem is upstream https://github.com/jupyterlab/jupyterlab/issues/7354 (bot auto-closed) and https://github.com/jupyter-widgets/ipywidgets/issues/1845.
@casperdcl So in vscode, does it also run the jupterlab?
vscode code was
bar = trange(epochs)
for epoch in bar:
for images in tqdm(loader, leave=False):
I am also struggling with this issue
I am also struggling with this issue
i found that fastprogress doesnt have this issue
TqdmCallback for Keras doesn't have this issue.
ipywidgets 7.0.0 widgetsnbextension 3.0.2
<div class="output_area"><div class="prompt"></div><div class="output_subarea jupyter-widgets-view"></div></div>
Looks like it deletes widget itself, not container. Show stopper when you have thousands of nested loops.