Closed JohnGuan closed 5 years ago
ntoll
commented
5 years ago @tmontes this didn't used to happen. Could it be something to do with the changes you made to OSX locale detection? (This is just a hunch based on what has/hasn't changed since the last release.)
ntoll
commented
5 years ago OK... here's what I've learned.
Previously, because of the odd way OSX does locale, it was falling back to "English" as the default language, so the print statement on line 66 was getting the string "Logging to ...". Now that Mu correctly detects OSX locale settings (thanks @tmontes), the print statement is getting the text translated into Chinese (presumably your locale): "记录日志到 /Users/johnguan/Library/Logs/mu/mu.log"
At this point, Python complains about encoding problems and breaks.
tmontes
commented
5 years ago @ntoll, thanks for pinging me. :)
I'll definitely try to have a look at this sometime later today. A few notes off the top of my head:
print function is failing at printing.zh_CN locale translated 'Logging to ...': '记录日志到 ...'.print failure results from failing to encode such a string to the sys.stdout encoding.sys.stdout encoding is determined and that, by itself, it is a non-trivial problem.sys.stdout.encoding to ASCII?Pseudo-reproduction example:
$ python3.6
Python 3.6.8 (v3.6.8:3c6b436a57, Dec 24 2018, 02:04:31)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdout.encoding
'UTF-8'
>>> s = '记录日志到'
>>> print(s) # works, `s` successfully encoded to UTF-8
记录日志到
>>> s.encode('UTF-8') # confirmation
b'\xe8\xae\xb0\xe5\xbd\x95\xe6\x97\xa5\xe5\xbf\x97\xe5\x88\xb0'
>>> s.encode('ASCII') # pseudo-reproduction
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-4: ordinal not in range(128)
>>>I get the same exact behaviour on CPython 3.7 here.
Suspicion: the CPython startup process is setting sys.stdout.encoding to something that can't encode the localised message. Needs further investigation which I'll try to do later today.
tmontes
commented
5 years ago Progress - Confirmed suspicion above.
What I did:
setup_logging just before the print(_('Logging to {}').format(LOG_FILE)) line:log.info('sys.stdout.encoding=%r', sys.stdout.encoding)2019-01-18 10:47:32,263 - root:66(setup_logging) INFO: sys.stdout.encoding='US-ASCII'open /Applications/Local/mu-editor.app. Log file contents:
2019-01-18 10:52:54,605 - root:66(setup_logging) INFO: sys.stdout.encoding='UTF-8'Possible quick fixes:
print things.print is useful, probably for diagnostic purposes, don't localize printed things."Real" fixes:
Plan on doing further investigation:
tmontes
commented
5 years ago @JohnGuan,
Do you think you could "hack" your mu installation -- with our guidance -- to see if we can make it work for you, when launched from the launchpad? This would be helpful in confirming the root cause of the issue on your system.
The idea would be:
Contents/Resources/app/mu/app.py inside the mu-editor.app bundle.print(_('Logging to {}').format(LOG_FILE))
to
print('Logging to {}'.format(LOG_FILE))log.info('sys.stdout.encoding=%r', sys.stdout.encoding)If this is something you feel you could try then great:
~/Library/Logs/mu/... should include a line with saying sys.stdout.encoding=...... Please share that.Otherwise, if this feels to complex don't worry. I'm sure we'll be able to make it work otherwise.
ntoll
commented
5 years ago @tmontes thank you! What you've just written on this issue is exactly my reading of the situation. I have already walked @JohnGuan through similar steps (although I omitted to ask for sys.stdout.encoding).
I believe the simplest solution is simply to remove the print statement. We only use print once and it's only there because I didn't remove it when I added the "log" pane to the UI (it helped sys admins at school find out what Mu was complaining about).
Ergo, I'll remove print and update the installers with the change.
tmontes
commented
5 years ago (now away from laoptop/code but...)
While you are at it, it may make sense to remove (or wrap in a try/except UnicodeEncodingError) any other print calls the code may have lying around... I have the feeling I saw at least one more. :)
ntoll
commented
5 years ago Folks, I've just updated the Mac package so it doesn't contain the problematic print statement. Ergo, I'll close this issue. Thanks for the report and efforts in diagnosing things.
ntoll
commented
5 years ago @tmontes I had a look at the use of print in the code base. We don't have any further uses which use gettext (i.e. all other print statements are in "contrib" modules which are outputting English ASCII informational messages.
tmontes
commented
5 years ago @ntoll that's good... :) But won't this one here, app.py:debug() be potentially problematic?
ntoll
commented
5 years ago How did I miss that? Doh. You have very sharp eyes.
Yes... I agree. That could be problematic. I'll re-open.
tmontes
commented
5 years ago ...I'm not sure under which circumstances the print in app.py:debug() would run -- IOW: how would it be missing the module to be debugged? Regardless of that, even if the argument isn't present, how would that print output reach a potential user? Is it captured somehow and displayed in a graphical pane? Or is the debugger supposed to be runnable from a command line interface as well, for more advanced users?
ntoll
commented
5 years ago The way the debugger works is described here: https://mu.readthedocs.io/en/latest/debugger.html#graphical-debugger
Basically, this entry point is used by the DebugRunner side of things to launch the code to be debugged. The output from this process is displayed in Mu's run pane - used to show and capture stdin/stdout.
Does this make sense?
I agree it may need a try/except around it though, for the same reason we removed the print statement.
Having said that, the ONLY way this block of code could be reached is if the user launched the DebugRunner "manually" from the command line WITHOUT specifying the python file to be debugged. Mu always launches the DebugRunner with a filename (it checks/complains if there's no filename), so "normal" users will never encounter this code branch. The only person on the planet who is ever likely to launch the DebugRunner "manually" is me, for development purposes, and I don't speak Chinese. ;-)
If you're happy that the chances of a Chinese speaking developer manually running the DebugRunner are sufficiently small (i.e. I think the chances are basically zero), then I'm happy to close this issue again. However, I can't help but smile to myself and think this sort of attitude is bound to come back and haunt me in the future. ;-)
tmontes
commented
5 years ago The way the debugger works is described here: https://mu.readthedocs.io/en/latest/debugger.html#graphical-debugger
Basically, this entry point is used by the DebugRunner side of things to launch the code to be debugged. The output from this process is displayed in Mu's run pane - used to show and capture stdin/stdout.
...thanks for the pointer and mini-overview. :)
I agree it may need a try/except around it though, for the same reason we removed the
Having said that, the ONLY way this block of code could be reached is if the user launched the DebugRunner "manually" from the command line WITHOUT specifying the python file to be debugged. Mu always launches the DebugRunner with a filename (it checks/complains if there's no filename), so "normal" users will never encounter this code branch. The only person on the planet who is ever likely to launch the DebugRunner "manually" is me, for development purposes, and I don't speak Chinese. ;-)
If you're happy that the chances of a Chinese speaking developer manually running the DebugRunner are sufficiently small (i.e. I think the chances are basically zero), then I'm happy to close this issue again. However, I can't help but smile to myself and think this sort of attitude is bound to come back and haunt me in the future. ;-)
My take:
print but skip the _(...) localisation: English perfectly fine for that.What do you say?
PS: It took me more time to write these words that it would have taken me/you implementing the change. :)
tjguk
commented
5 years ago FWIW I agree with @tmontes in principle.
But also I'd add what is doubtless obvious to both of you: that having the discussion and thought process "out loud" as it were is so helpful later when you can't quite remember what process you went through to get to that point. (Even if you dutifully put good comments in place).
That's where I'm haltingly trying to go with my design notes idea:
ntoll
commented
5 years ago +1 to the assessment, discussion and suggested solution. Also +1 to public discussion about such things. Thank you @tmontes and @tjguk. I'll fix right now.
tmontes
commented
5 years ago Thanks @ntoll + @tjguk for your inputs and guidance. :)
Can't open directly from Launchpad on macOS, but can start in Terminal. macOS version: 10.14.2 (18C54)