Display coding problem

[dcl-lily]

Code：

-- coding:utf-8 --

import better_exceptions import sys print(sys.stdout.encoding) better_exceptions.hook()

foo = 52

def shallow(a, b): deep(a + b)

def deep(val): global foo assert val > 10 and foo == 60

bar = foo - 50 shallow(bar, 15) shallow(bar, 2)

display: "C:\Program Files\Python36\python.exe" C:/Users/Alex/PycharmProjects/Test/Test02.py UTF-8 Traceback (most recent call last): File "C:/Users/Alex/PycharmProjects/Test/Test02.py", line 20, in shallow(bar, 15) �� �� 2 �� <function shallow at 0x000001860AAEF730> File "C:/Users/Alex/PycharmProjects/Test/Test02.py", line 11, in shallow deep(a + b) �� �� �� 15 �� �� 2 �� <function deep at 0x000001860B079620> File "C:/Users/Alex/PycharmProjects/Test/Test02.py", line 16, in deep assert val > 10 and foo == 60 �� �� 52 �� 17 AssertionError: assert val > 10 and foo == 60

Process finished with exit code 1

system: windows 10 python: 3.6

[Delgan]

Hi @dcl-lily

The errors are written to the stderr, could you please tell us the value of sys.stderr.encoding?

Also, what is the output of the chcp command on your Windows terminal?

[dcl-lily]

hi
sys.stderr.encoding value UTF-8

PS C:\Users\Alex> chcp 活动代码页: 936 PS C:\Users\Alex> python -m better_exceptions Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. (BetterExceptionsConsole)

a = 1 b = 'fgh' c = a + b Traceback (most recent call last): File "", line 1, in c = a + b �� �� 'fgh' �� 1 TypeError: unsupported operand type(s) for +: 'int' and 'str' print(sys.stderr.encoding) Traceback (most recent call last): File "", line 1, in print(sys.stderr.encoding) NameError: name 'sys' is not defined import sys print(sys.stderr.encoding) utf-8 print(sys.stdin.encoding) utf-8 print(sys.stdout.encoding) utf-8

think

[Delgan]

@dcl-lily Thanks for the output.

I am afraid I do not know what is causing the problem. This looks like an issue with the Windows 10 terminal, but I am not able to reproduce it.

Some investigations idea I found on Google:

• Run python test.py > file.txt and check with a text editor that file.txt is correct
• Change the font of your Windows terminal (try them all)
• Execute the command chcp 65001 before your Python script
• Try using the legacy console
• Launch your script with a Python version < 3.6 (PEP 528)

I will see if I can find some time in the next days to dig a bit deeper about this issue.

[dcl-lily]

hi used python 3.5 display OK, in version 3.6 to Change the system configuration but not be able to display normally

[chen19901225]

I get the same problem env: windows7 python: python3.7 chcp: "活动代码页: 936" "sys.stderr.encoding": "utf-8", "locale.getpreferredencoding() ": "cp936", So the problem seem this file "better_exception/encoding.py" line 10 ENCODING = locale.getpreferredencoding(), So I replace with ENCODING = sys.getdefaultencoding() and it works hope it works for you too

[dcl-lily]

@chen19901225 think，is good

[Qix-]

@Delgan can you investigate whether or not https://github.com/Qix-/better-exceptions/issues/53#issuecomment-387991544 is a proper solution or a hack? We might need to change it.

[Delgan]

@Qix- Sure, I will take a look this week-end. ;)

[Delgan]

Hey @chen19901225 and @dcl-lily

We would like to merge your fix "upstream", so you can enjoy future releases of better_exceptions without having to maintain your own local copy of the modified library.

As I'm still not able to reproduce the issue, would you please help me by answering a few questions about your configuration?

• What font and size are used in the properties of your Windows terminal (make a screen if you wish)?
• Does your fix replace mojibake characters with proper utf8 └ or with ascii -> fallback?
• What is the output of sys.getdefaultencoding()?

Thanks!

[Qix-]

Closing due to insufficient repro information. Feel free to comment back if you want to add something and we can re-open as necessary.

Thanks for following up @Delgan.

[frostming]

This solution disables the debugging feature in IPython, which complains no traceback is found,

[bytemain]

I meet this problem too. Window 10 System language: Chinese

---

so I decide to try #issuecomment-387991544's solution,

and it works.

• sys.getdefaultencoding(): >>> utf-8
• locale.getpreferredencoding() >>> cp936

[Delgan]

Thanks for the report @lengthmin!

I'm re-opening this issue, then, since it's clear something should be done on our side to fix it. I will try to find some time to investigate this.

Which Python version are you using please?

[bytemain]

Python 3.7.1 I think the code locale.getpreferredencoding() is to use every country's own encoding type, but most of them don’t support every unicode character completely. so it better to switch to UTF-8

[Delgan]

As we are writing exceptions on sys.stderr, I thought we may use sys.stderr.encoding. Does it equal to utf8 on your system?

[bytemain]

yes

[Delgan]

Great, thanks!

@Qix- Any thoughts on replacing locale.getpreferredencoding() with sys.stderr.encoding? Can you remember why getpreferredencoding() was used in the first place?

sys.getdefaultencoding() could work too, but it's seems to contain the encoding internally used by CPython to store str objects, so this is not much related to terminal output.

[Qix-]

If it works, then let's do it :)

[herrome]

change the code file "ENCODING = locale.getpreferredencoding()" to "ENCODING = utf-8" it works

[Qix-]

@herrome It works for you, but doesn't for anyone that is using a different encoding. I wish we lived in a world where utf-8 was used everywhere but that just isn't the case yet.