search

pytorchbearer / torchbearer

torchbearer: A model fitting library for PyTorch
http://pytorchbearer.org
MIT License
636 stars 65 forks source link

Tqdm for Jupyter Notebook #305

Closed kl2792 closed 6 years ago

kl2792 commented 6 years ago

Each iteration of TQDM starts a new line in Jupyter Notebook -- is there any way to integrate one of the suggested fixes into torchbearer?

(ref: https://github.com/tqdm/tqdm/issues/375, https://stackoverflow.com/a/47200571)

kl2792 commented 6 years ago

Restarting the notebook temporarily fixed the problem, but it looks like 1 extra newline is added for every fit_generator call.

On the 1st fit_generator call, it works fine.

On the 2nd fit_generator call:

0/1000(t):   0%|          | 0/1000 [00:00<?, ?it/s]
0/1000(t):   0%|          | 1/1000 [00:00<07:57,  2.09it/s]
0/1000(t):   0%|          | 1/1000 [00:00<07:57,  2.09it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   0%|          | 2/1000 [00:00<07:24,  2.24it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   0%|          | 2/1000 [00:00<07:24,  2.24it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   0%|          | 3/1000 [00:01<07:48,  2.13it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   0%|          | 3/1000 [00:01<07:48,  2.13it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   0%|          | 4/1000 [00:01<07:17,  2.28it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   0%|          | 4/1000 [00:01<07:17,  2.28it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   0%|          | 5/1000 [00:02<06:48,  2.44it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   0%|          | 5/1000 [00:02<06:48,  2.44it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 6/1000 [00:02<06:36,  2.51it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 6/1000 [00:02<06:36,  2.51it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 7/1000 [00:02<06:17,  2.63it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 7/1000 [00:02<06:17,  2.63it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 8/1000 [00:03<05:59,  2.76it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 8/1000 [00:03<05:59,  2.76it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 9/1000 [00:03<05:58,  2.76it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 9/1000 [00:03<05:58,  2.76it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 10/1000 [00:03<05:32,  2.98it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 10/1000 [00:03<05:32,  2.98it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 11/1000 [00:04<05:37,  2.93it/s, running_loss=7.09, running_acc=0.632]
0/1000(t):   1%|          | 11/1000 [00:04<05:37,  2.93it/s, running_loss=5.01, running_acc=0.113]
0/1000(t):   1%|          | 12/1000 [00:04<05:34,  2.96it/s, running_loss=5.01, running_acc=0.113]
...

On the third:


0/1000(t):   0%|          | 0/1000 [00:00<?, ?it/s]

0/1000(t):   0%|          | 1/1000 [00:00<07:21,  2.26it/s]

0/1000(t):   0%|          | 1/1000 [00:00<07:21,  2.26it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   0%|          | 2/1000 [00:00<06:59,  2.38it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   0%|          | 2/1000 [00:00<06:59,  2.38it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   0%|          | 3/1000 [00:01<07:29,  2.22it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   0%|          | 3/1000 [00:01<07:29,  2.22it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   0%|          | 4/1000 [00:01<07:03,  2.35it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   0%|          | 4/1000 [00:01<07:03,  2.35it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   0%|          | 5/1000 [00:02<06:40,  2.49it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   0%|          | 5/1000 [00:02<06:40,  2.49it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 6/1000 [00:02<06:32,  2.54it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 6/1000 [00:02<06:32,  2.54it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 7/1000 [00:02<06:11,  2.67it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 7/1000 [00:02<06:11,  2.67it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 8/1000 [00:03<05:55,  2.79it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 8/1000 [00:03<05:55,  2.79it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 9/1000 [00:03<05:54,  2.80it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 9/1000 [00:03<05:54,  2.80it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 10/1000 [00:03<05:28,  3.02it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 10/1000 [00:03<05:28,  3.02it/s, running_loss=3.67, running_acc=0.085]

0/1000(t):   1%|          | 11/1000 [00:04<05:34,  2.96it/s, running_loss=3.67, running_acc=0.085]

And so on. Is there some callback that's supposed to be removed on KeyboardInterrupt? Happy to put in a PR to fix this behavior, but does anyone know why it's occurring?

MattPainter01 commented 6 years ago

I'm not sure why this behaviour is happening myself, any ideas @ethanwharris?

The tqdm_notebook fix looks simple enough to implement, I'll look into it on Monday and hopefully it will fix this.

Although feel free to put in a pr for this in the mean time.

ethanwharris commented 6 years ago

Seems like the issue is just tqdm not handling notebooks very well. Simplest idea is to have a tqdm_notebook callback which uses the notebook variant. Or perhaps, allow the tqdm callback to be given the tqdm module to use, assuming the commands are the same. I prefer the second option if possible. Thoughts?

MattPainter01 commented 6 years ago

Yeah, I agree the second option is better. Not sure how we decide to integrate it with the verbose flag though, unless we change verbose to "printer_type" or something. Could still keep the integer flags as in verbose but also allow strings.

Alternatively we allow a printer callback to be passed and just default it to Tqdm. So then arguments such as notebook could be given when the callback is initialised.

MattPainter01 commented 6 years ago

This should now be fixed in master, the callback list was persisting through the fit calls (which wasn't the original intention) leading to multiple Tqdm callbacks being added and running simultaneously, thus the new lines.

@kl2792 Tqdm now works properly for multiple fit calls in my local jupyter, could you verify if this works for you too? Thanks!

kl2792 commented 6 years ago

Works for me, thanks so much!