Windows error 6 when subprocess' stdout is redirected to os.devnull

[osvenskan]

Background: I'm using version 0.4. I have win-unicode-console installed as a Python patch in sitecustomize.py. I only have one Python installed, and it's Anaconda Python 2.7.10.

I have a complicated piece of code that breaks when run with win-unicode-console enabled. I can reproduce the problem with a small test case (see attachments). On their own, the test cases are strange. Please keep in mind that they're a snippet of a much larger piece of code. :smiley:

I think the part of my scenario that confuses win-unicode-console is that I'm redirecting a subprocess' stdout to os.devnull. This is key to the problem, because if I redirect it to a temp file instead, the problem goes away. I guess os.devnull does not mimic a real file in some respect.

Attached are hello_world.py, fails.py, and succeeds.py. The last two scripts invoke hello_world which just prints to the console. The only difference between fails and succeeds is that fails pipes its subprocess' stdout to os.devnull, while succeeds uses a temp file.

The attachments have a .txt file extension to keep github happy. Remove that before running the files.

[osvenskan]

Aaaargh, attachments rejected again. Here they are inline.

hello_world.py:

print "No one expects the Spanish Inquisition!"

fails.py:

import subprocess
import os
import sys

with open(os.devnull, 'w') as dev_null:
    proc = subprocess.Popen('python hello_world.py',
                            stdout=dev_null,
                            stderr=subprocess.PIPE,
                            shell=True)

    while True:
        # read output
        out = proc.stderr.read(1)
        if out:
            sys.stdout.write(out)
            sys.stdout.flush()

        # end of process
        if out == '' and proc.poll() != None:
            break

    proc.communicate()

succeeds.py:

import subprocess
import os
import sys
import tempfile

with tempfile.TemporaryFile() as dev_null:
    proc = subprocess.Popen('python hello_world.py',
                            stdout=dev_null,
                            stderr=subprocess.PIPE,
                            shell=True)

    while True:
        # read output
        out = proc.stderr.read(1)
        if out:
            sys.stdout.write(out)
            sys.stdout.flush()

        # end of process
        if out == '' and proc.poll() != None:
            break

    proc.communicate()

[osvenskan]

I forgot to mention (last comment for now, I promise!) that this is the error from fails.py:

> python .\fails.py
Traceback (most recent call last):
  File "hello_world.py", line 1, in <module>
    print "No one expects the Spanish Inquisition!"
  File "C:\Users\ond\Anaconda\lib\site-packages\win_unicode_console\streams.py", line 219, in write
    self.base.write(s)
  File "C:\Users\ond\Anaconda\lib\site-packages\win_unicode_console\streams.py", line 179, in write
    return self.base.write(s)
  File "C:\Users\ond\Anaconda\lib\site-packages\win_unicode_console\streams.py", line 128, in write
    raise OSError(self._error_message(GetLastError()))
OSError: Windows error 6

[Drekin]

In the case that a stream is redirected win-unicode-console should do nothing with the stream – it doesn't lead to console so there is no problem win-unicode-console should fix. The problem is how to detect whether a stream is redirected. The current approach compares fileno and isatty. The problem is it doesn't work with nul (as I found out due to your report) since its isatty returns True.

The particular error we get is “invalid handle”. It is the same as if you tried to write to win_unicode_console.streams.stdout_text (and then flushed) when stdout it redirected.

[Drekin]

Check out the newest development version where this should be fixed.

[osvenskan]

The latest development version fixes the original problem I saw in the complicated, non-test case code. Thanks!

[Drekin]

What do you mean? When a stream is redirected, there is nothing to be fixed by win-unicode-console.

[Drekin]

This was fixed, just download the latest development version.