on python3, print() does not automatically flush() stdout/stderr

[ronaldoussoren]

Original report by Cosimo Lupo (Bitbucket: anthrotype, GitHub: anthrotype).

---

Hi,

I noticed that doing print() from a PyObjC application behaves differently when run from python 2 or python 3.

I tried to make a standalone .app with py2app from the example "HelloWorld.py" script:

https://bitbucket.org/ronaldoussoren/pyobjc/src/9bdbc79f0a19b0c19a9aa17a7a0ea74fbd18667d/pyobjc-core/Examples/Scripts/HelloWorld.py

The setup.py looks like this:

from setuptools import setup

setup(
    app=["HelloWorld.py"],
    setup_requires=["py2app"],
)

Now, if I build it with Python 3.5.2 (downloaded from python.org), using pyobjc 3.2.1 (released today) and the latest py2app, the output of print() is only shown in the Console.app only when the application is closed.

Similarly, If I run it from the Terminal.app by doing dist/HelloWorld.app/Contents/MacOS/HelloWorld, the output is only printed after quitting.

However, if I manually do sys.stdout.flush() after each print, then I see the output immediately after clicking the buttons, while the app is still running.

The issue does not occur when compiling the same code with the system python 2.7.10 and same version of pyobjc 3.2.1 (in a virtualenv).

Thank you for your support.

Cosimo

[ronaldoussoren]

Original comment by Cosimo Lupo (Bitbucket: anthrotype, GitHub: anthrotype).

---

Well.. that's the way the built-in print function works apparenty: https://docs.python.org/3/library/functions.html#print

I need to pass flush=True otherwise output is buffered according to the sys.stdout buffer size.

BTW, if I use logging instead of print(), it flushes immediately after each message in both 2 and 3.

Sorry for the noise.

[ronaldoussoren]

Original comment by Cosimo Lupo (Bitbucket: anthrotype, GitHub: anthrotype).

---

according to the docs, the standard streams are line-buffered only when running from an interactive console, and block-buffered when redirected to a file.

https://docs.python.org/3/library/sys.html#sys.stdout

I still don't understand why this would work in Python 2 and not in Python 3, though..

Also, I wonder if I could pass the PYTHONUNBUFFERED variable or the -u command line switch to a py2app-generated app so that it flushes stdout immediately?

[ronaldoussoren]

Original comment by Cosimo Lupo (Bitbucket: anthrotype, GitHub: anthrotype).

---

Could anybody tell me where in pyobjc source code such redirection from Python's sys.stdout to the Console.app actually takes place?

[ronaldoussoren]

Original comment by Ronald Oussoren (Bitbucket: ronaldoussoren, GitHub: ronaldoussoren).

---

PyObjC doesn't perform the redirection.

On OSX 10.8 or earlier the system automatically redirected stdout and stderr to the Console.app logs, on newer OSX releases this doesn't happen but py2app contains code to perform the redirection using the ASL library, the relevant code is in the main.c for the app template.

That code likely causes the problem, it replaces the file descriptors for stdout and stderr by new ones provided by ASL. In python2 sys.stdout (used by print) uses the C "FILE*" stream for stdout, which is initialised before the file descriptor trickery by py2app and hence doesn't notice that stdout isn't a tty anymore; in python3 sys.stdout is fully implemented in Python (with a C accelerator) and is initialised after py2app changes the file descriptors and probably notices that those file descriptors aren't ttys.

I guess I can teach py2app to set "PYTHONUNBUFFERED" in the environment before initialising the Python interpreter, that would fix your problem.

[ronaldoussoren]

Original comment by Cosimo Lupo (Bitbucket: anthrotype, GitHub: anthrotype).

---

That would be great! Thanks for the detailed explanation!

[ronaldoussoren]

Original comment by Ronald Oussoren (Bitbucket: ronaldoussoren, GitHub: ronaldoussoren).

---

I've committed an update for this in the py2app repository (changeset 3f8fca9f1158)

P.S. the link to the changeset is wrong, it links to a (probably non-existent) changeset in the PyObjC repository instead of the py2app one.

[ronaldoussoren]

Original comment by Cosimo Lupo (Bitbucket: anthrotype, GitHub: anthrotype).

---

thanks a lot!

[ronaldoussoren]

Original comment by Ronald Oussoren (Bitbucket: ronaldoussoren, GitHub: ronaldoussoren).

---

Removing version: 3.1 (automated comment)