No output in Google colab terminal commands

[theosanderson]

Thanks for this utility.

Something I noticed when trying to test something that used it, is that when terminal-like commands that use alive_progress are run in Google Colab cells (by prefixing with !), alive_progress does not provide any output at all, even on completion.

Here is a reproducible example: https://colab.research.google.com/drive/1swUZNMAIY_XVGhJbpB9zLxvjH7391CM8?usp=sharing

It just seemed odd, as generally these calls work normally for a whole range of things, including tqdm bars, so I thought I'd raise the Issue. But I guess it's probably some complex thing about how one or other is implemented, so don't worry if nothing can be done about it.

Thanks

[rsalmei]

Thanks, @theosanderson.

I'm not aware of how this google colab works, but it does seem like an ipython.

I'd say it does work the way you've called it, but colab is not correctly capturing the terminal output... Look what I got:

It was curious how the lines were being skipped in a timely fashion as I'd expect, just without any output on the browser, but it surely was running in the back.

Anyway, I really don't know how to help with this. tqdm has a whole different implementation for ipython, using graphical widgets, which can not really be compared. I think it is related to how colab creates new shell processes, and capture their stdout and stderr outputs...

For me, the most important is that direct code like this works in real-time:

https://user-images.githubusercontent.com/6652853/165472062-67425e6f-a77b-4b77-a575-85edc530ad96.mov

I'm going to close this one, but feel free to reply if you need anything else! 👍

[theosanderson]

Hi there,

(Thanks for the response!)

I've now looked into this, and it is because shutil.get_terminal_size()[0] returns 0 in this context. So I can rescue things by forcing def cols to return say 80 in TTY if shutil.get_terminal_size()[0]==0. Do you see a downside of behaviour like that? I can't see a real life scenario where one is attached to a TTY terminal which actually has zero columns (but I don't know much about this area).

Thanks

[rsalmei]

Hey, are you kidding??? 😱 Wow, that is weird!

I've used to use os.get_terminal_size(), which is more low-level and way faster, but it was failing in exoteric environments just like this one, so I've changed to the shutil one, which protects against any errors, and just returns a default (80, 24) in case anything goes wrong. This means that colab is actively returning ZERO here, which is bizarre in my view...

Errr, are you sure? I just had to see that, and it didn't behave like you've said:

[theosanderson]

Like I said, it generally works in colab cells running normal python code, and that is as you say because there shutil returns a normal size. What fails is when you do a call to a terminal command in a cell, by prefixing with !. E.g. !python -m alive_progress.tools.demo

And yes, I can confirm that the reason that fails is that it actively receives 0,0. I tested the fix, and it works, and I also confirmed that if you call !python print_shutils_size.py on a simple script written to print that it returns 0,0

[rsalmei]

Ohh, I see... It's very weird anyway, even a non-tty terminal does report its size 🤔

Well, I'm glad your static columns fix does work, but I don't think it warrants a feature to support this. Setting the number of columns in any other environment can have awful adverse effects, like truncating lines or uncontrolled scrolling by spamming the terminal.

[theosanderson]

I did a bit more research and terminals of size zero do seem an issue in several environments (https://github.com/python/mypy/pull/9651, https://github.com/aws/aws-cdk/issues/2253, https://github.com/Textualize/rich/blob/v9.1.0/rich/console.py#L669-L672), so I would probably have ended up doing something in a fork for my own use, however if I understand correctly this will no longer be an issue in Python 3.11 thanks to a shutil fix:

https://github.com/python/cpython/issues/86340

[theosanderson]

(Thanks for considering!)

[rsalmei]

Wow man, you really did your homework! I've seen the links, and I see your point, it really is a bug in Python, and perhaps worth considering. I'm glad they'll fix it in 3.11, but how unfortunate not back-port it!! 😞 Well, I'll think about including it! 👍