py.test creates an environment that causes colorama to fail

[GoogleCodeExporter]

When using py.test to run a file that imports colorama on Windows, the import 
fails (as far as I can tell, because the Windows standard handle does not point 
to a proper console)
Here's an example:

>type t.py
from pip.vendor import colorama
>python t.py >a
>type a
>py.test t.py
============================================= test session starts 
============================================== platform win32 -- Python 3.3.0 
-- pytest-2.4.1
collected 0 items / 1 errors

==================================================== ERRORS 
==================================================== 
____________________________________________ ERROR collecting t.py 
_____________________________________________ t.py:1: in <module>
>   from pip.vendor import colorama
pip\__init__.py:10: in <module>
>   from pip.log import logger
pip\log.py:11: in <module>
>   from pip.vendor import colorama
pip\vendor\colorama\__init__.py:1: in <module>
>   from .initialise import init, deinit, reinit
pip\vendor\colorama\initialise.py:4: in <module>
>   from .ansitowin32 import AnsiToWin32
pip\vendor\colorama\ansitowin32.py:11: in <module>
>       winterm = WinTerm()
pip\vendor\colorama\winterm.py:25: in __init__
>       self._default = 
win32.GetConsoleScreenBufferInfo(win32.STDOUT).wAttributes
pip\vendor\colorama\win32.py:64: in GetConsoleScreenBufferInfo
>           handle, byref(csbi))
E       ctypes.ArgumentError: argument 2: <class 'TypeError'>: expected 
LP_CONSOLE_SCREEN_BUFFER_INFO instance instead of pointer to 
CONSOLE_SCREEN_BUFFER_INFO
=========================================== 1 error in 0.74 seconds 
===========================================

I'm not sure if this is a py.test bug or a colorama bug, so I have reported it 
to both projects. I suspect that maybe colorama should ensure on import that it 
does not try to initialise itself if the standard output is not suitable.

Original issue reported on code.google.com by p.f.mo...@gmail.com on 3 Oct 2013 at 2:04

[GoogleCodeExporter]

Thanks for the bug report.

It was my *intent* when I wrote Colorama that it should do nothing on import, 
until you call 'init', so I'm surprised that just importing colorama can cause 
a problem like this. I believe you though, so it looks like maybe I failed in 
that regard.

This is pure speculation, but this reminds me of a recent issue that was fixed 
under 64 bit Windows by changing the ctypes definitions used throughout 
colorama. If you have time and are keen to progress this issue, you might like 
to try to reproduce using an older version of colorama 
(https://pypi.python.org/pypi/colorama/0.2.6) and report back here whether you 
could reproduce or not. Then we'll know for sure whether or not the changes in 
v0.2.7 introduced this problem. Thanks!

Original comment by tart...@gmail.com on 3 Oct 2013 at 2:27

[GoogleCodeExporter]

The issue was actually found with the version of colorama that is bundled with 
pip (which is version 0.2.5).

Original comment by p.f.mo...@gmail.com on 3 Oct 2013 at 2:30

[GoogleCodeExporter]

Aha! Good to know. Thanks.

My intent is that if you 'pip install colorama' you'll actually get v0.2.7 
these days. (or use "pip install -U colorama" if you already have colorama 
installed, to tell pip to update it)

Let me know if the issue is still present *after* the update then! :-)

Thanks.

Original comment by tart...@gmail.com on 3 Oct 2013 at 2:35

[GoogleCodeExporter]

Sorry, I should have put everything together. Yes, it's still present with 
0.2.7.

A comment from the py.test developer: "pytest uses py.io.TerminalWriter() which 
implements coloring via ctypes call". I suspect that there's some nasty 
interaction between the two packages messing with the console somehow.

Original comment by p.f.mo...@gmail.com on 3 Oct 2013 at 2:40

[GoogleCodeExporter]

The py.test developers have fixed this (by using colorama to provide coloured 
output rather than the py.io code, as it happens :-)) so it looks like the 
problem has been solved.

Original comment by p.f.mo...@gmail.com on 3 Oct 2013 at 3:20

[GoogleCodeExporter]

HoooRAY! Thanks very much for chasing up and for reporting back, that's very 
much appreciated.

Original comment by tart...@gmail.com on 3 Oct 2013 at 3:33

• Changed state: WontFix

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

:( I'm having the same problem.  I need to use ctypes and 
windll.kernel32.GetConsoleScreenBufferInfo(h, csbi) to get the terminal/console 
size for layout reasons.  But when using with colorama I get a kaboom instead.  
I am going to try to reorder the calls, but it would be nice to know what the 
reason for this is and how to fix it.

I could have sworn this used to work.

Original comment by geekad...@mgmiller.net on 18 Jan 2014 at 8:10

[GoogleCodeExporter]

It's possible it did used to work. I've made and accepted a few commits over 
the last couple of years which might have broken it.

I'll try and run a bisect to see whether I can spot which commit broke it.

Original comment by tart...@gmail.com on 21 Jan 2014 at 7:43

[GoogleCodeExporter]

...or anyone else could, if they fancied helping out. Thx.

Original comment by tart...@gmail.com on 21 Jan 2014 at 7:43

[GoogleCodeExporter]

Original comment by tart...@gmail.com on 21 Jan 2014 at 7:44

• Changed state: Accepted

[GoogleCodeExporter]

So just to be explicit, the problem is that calling 
"windll.kernel32.GetConsoleScreenBufferInfo(h, csbi)" via ctypes AFTER 
colorama.init() causes a crash? (but before is ok.)

Original comment by tart...@gmail.com on 21 Jan 2014 at 7:45

[GoogleCodeExporter]

Having pyreadline (which uses the console APIs) installed also seems to trigger 
this issue. See https://github.com/pyreadline/pyreadline/issues/21 and 
https://github.com/pyreadline/pyreadline/issues/18 and 
https://github.com/pypa/pip/issues/1580.

Original comment by p.f.mo...@gmail.com on 19 Feb 2014 at 2:09

[GoogleCodeExporter]

Hmm. I think p.f.moore is right, this issue does sound more likely to be 
related and my musings above.

I can reproduce this when I run 'from pip.vendor import colorama', but I can't 
reproduce it by importing the latest colorama (which has just tonight had the 
pyreadline incompatibility fixed.) Unfortunately, I can't reproduce it by 
importing the commit right before the pyreadline incompatibility was fixed.

Maybe it was fixed by some other recent change? I'll try against some older 
versions...

Original comment by tart...@gmail.com on 17 Apr 2014 at 10:02

• Changed state: Started

[GoogleCodeExporter]

See https://github.com/pyreadline/pyreadline/pull/24

The issue is that if you apply argtypes to something you're referencing as 
kernel32.XXX you get a shared copy. If your argtypes are too restrictive, this 
causes issues for other applications.

I don't know whether colorama needs a similar fix.

Original comment by p.f.mo...@gmail.com on 17 Apr 2014 at 10:40

[GoogleCodeExporter]

@p.f.moore. Thanks for the advice.

So the fix for colorama's incompatibility with pyreadline turned out to be to 
replace this:

    from ctypes import windll
    stdHandle = windll.kernel32.GetStdHandle

with this:

    import ctypes
    windll = LibraryLoader(ctypes.WinDLL)
    stdHandle = windll.kernel32.GetStdHandle

This apparently means I now have my own instance of windll, which can happily 
be incompatible with any other usage, and will cause no problems.

I'm at a loss as to whether that is expected to also fix this issue.

Can anyone else who has reported this issue still reproduce it against v0.3.0? 
I am not able to reproduce it.

Testing against significantly older versions is proving problematic since I 
lost the history when migrating to git because my Mercurial repo appeared to be 
corrupted (fresh checkouts could not display more than the first few entries of 
a log with diffs before giving 'no such node' errors after exiting.)

Does anyone know? Are there any other attributes of ctypes other than 'windll' 
which I should convert in a similar way? I'm going to go read the ctypes docs 
now, but when  I tried this the other night (tired), it didn't shed much light 
for me.

I'm likely to mark it as fixed unless anyone pipes up to contradict. Thanks for 
any input.

Original comment by tart...@gmail.com on 19 Apr 2014 at 11:15

[GoogleCodeExporter]

Migrated to https://github.com/tartley/colorama/issues/40
Closing as 'duplicate'

Original comment by tart...@gmail.com on 16 Feb 2015 at 12:09

• Changed state: Duplicate