Escape unencodable messages

[kri-k]

PR for #260

Problem: teamcity-messages plugin fails in cases when the sys.stdout encoding is unable to handle passed symbols. This mostly happens on Windows hosts due to the behavior, described in the documentation:

On Windows, UTF-8 is used for the console device. Non-character devices such as disk files and pipes use the system locale encoding (i.e. the ANSI codepage).

How to reproduce: Let's say we have a test.py file with the following content:

import sys
import unicodedata
def test_bad_encoding():
    bad_symbol = unicodedata.lookup('REPLACEMENT CHARACTER')
    sys.stdout.write(bad_symbol)

If we run it on Windows

python3 -m pytest test.py --teamcity > test_log

then most probably encoding of the sys.stdout will be cp1252 or something like that. And test_log will contain the following error:

<...>
PASSED                                        [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "...\site-packages\_pytest\main.py", line 269, in wrap_session
<...>
INTERNALERROR>   File "...\site-packages\teamcity\messages.py", line 187, in testStdOut
INTERNALERROR>     self.message('testStdOut', name=testName, out=out, flowId=flowId)
INTERNALERROR>   File "...\site-packages\teamcity\messages.py", line 101, in message
INTERNALERROR>     retry_on_EAGAIN(self.output.write)(self.encode(message))
INTERNALERROR>   File "...\site-packages\teamcity\messages.py", line 68, in encode
INTERNALERROR>     value = value.encode(self.encoding)
INTERNALERROR>   File "...\encodings\cp1252.py", line 12, in encode
INTERNALERROR>     return codecs.charmap_encode(input,errors,encoding_table)
INTERNALERROR> UnicodeEncodeError: 'charmap' codec can't encode character '\ufffd' in position 132: character maps to <undefined>

What does this pull request do: Let's look how bad test output is handled by the pytest itself. Let's add assert False line to the test and look at the result: python3 -m pytest test.py > test_log

test.py F                                                                [100%]

================================== FAILURES ===================================
______________________________ test_bad_encoding ______________________________

    def test_bad_encoding():
        bad_symbol = unicodedata.lookup('REPLACEMENT CHARACTER')
        sys.stdout.write(bad_symbol)
>       assert False
E       assert False

test.py:6: AssertionError
---------------------------- Captured stdout call -----------------------------
\ufffd
=========================== short test summary info ===========================
FAILED test.py::test_bad_encoding - assert False
============================== 1 failed in 0.08s ==============================

Captured stdout is escaped. This pull request is for the teamcity-plugin to similarly escape unencodable characters.

[kri-k]

Pypy 3 (Linux) Result: | Tests failed: 14 (14 new), passed: 123, ignored: 26

I'm pretty sure it's not my fault 😃 master branch is also broken, as I can see

[mikekidya]

I'm pretty sure it's not my fault 😃 master branch is also broken, as I can see

Yes, looking into it. Looks like there are some failures at new PyPy version (

Anyway, changes looks safe to me, let's merge them Thank you for you change!

[kri-k]

@mikekidya I'm glad I could help with this bugfix. BTW, are there any deadlines when the new version of package will be released on pypi? It would be great if a new version released soon :)