encoding error when joining path

[62a07a1b-c248-4c64-9f7a-30815f696946]

BPO	509117
Nosy	@tim-one, @loewis, @warsaw
Files	msvc.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['library'] title = 'encoding error when joining path' updated_at = user = 'https://bugs.python.org/pointal' ``` bugs.python.org fields: ```python activity = actor = 'loewis' assignee = 'none' closed = True closed_date = None closer = None components = ['Distutils'] creation = creator = 'pointal' dependencies = [] files = ['313'] hgrepos = [] issue_num = 509117 keywords = [] message_count = 9.0 messages = ['9009', '9010', '9011', '9012', '9013', '9014', '9015', '9016', '9017'] nosy_count = 4.0 nosy_names = ['tim.peters', 'loewis', 'barry', 'pointal'] pr_nums = [] priority = 'normal' resolution = 'fixed' stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue509117' versions = [] ```

[62a07a1b-c248-4c64-9f7a-30815f696946]

[platform] Problem on Python 2.1.1 and Python 2.2. Platform Windows 2000 SP2, french version. With MSVC++6 SP4.

[seealso] It looks like to be a non-us platform problem, see: http://mail.python.org/pipermail/python-list/2002- January/083903.html

[description]
While installing 
StandaloneZODB, python setup.py build ended with the followiong 
traceback:
...[zipped many file copying & Co]...
copying 
zdaemon\ZDaemonLogging.py -> build\lib.win32-
2.1\zdaemon
copying zdaemon\__init__.py -> build\lib.win32-
2.1\zdaemon
running build_ext
Traceback (most recent call 
last):
  File "setup.py", line 108, in ?
    headers = 
['ExtensionClass/src/ExtensionClass.h', 
'ZODB/cPersistence.h'],
  File 
"t:\dev\python21\lib\distutils\core.py", line 138, in setup

dist.run_commands()
  File 
"t:\dev\python21\lib\distutils\dist.py", line 899, in 
run_commands
    self.run_command(cmd)
  File 
"t:\dev\python21\lib\distutils\dist.py", line 919, in 
run_command
    cmd_obj.run()
  File 
"t:\dev\python21\lib\distutils\command\build.py", line 106, 
in run
    self.run_command(cmd_name)
  File 
"t:\dev\python21\lib\distutils\cmd.py", line 328, in 
run_command
    self.distribution.run_command(command)
  File 
"t:\dev\python21\lib\distutils\dist.py", line 919, in 
run_command
    cmd_obj.run()
  File 
"t:\dev\python21\lib\distutils\command\build_ext.py", line 
230, in run
    force=self.force)
  File 
"t:\dev\python21\lib\distutils\ccompiler.py", line 957, in 
new_compiler
    return klass (verbose, dry_run, force)
  File 
"t:\dev\python21\lib\distutils\msvccompiler.py", line 220, 
in __init__
    os.environ['path'] = string.join(path,';')

File "t:\dev\python21\lib\string.py", line 128, in join

return sep.join(words)
UnicodeError: ASCII decoding error: 
ordinal not in range(128)

[61337411-43fc-4a9c-b8d5-4060aede66d0]

Logged In: YES user_id=21627

Can you please enhance your local copy of distutils/msvccompiler to add a line of

print repr(path)

immediately above line 220, and report the output?

[62a07a1b-c248-4c64-9f7a-30815f696946]

Logged In: YES user_id=440351

Here is the "print repr(path)" result:

[u'T:\\dev\\MSVS\\Common\\MSDev98\\Bin', u'T:\\dev\\MSVS\\VC98\\BIN', u'T:\\dev\\MSVS\\Common\\TOOLS', u'T:\\dev\\MSVS\\Common\\ TOOLS\\WINNT', u'C:\\WINNT\\system32', u'C:\\WINNT', u'C:\\WINNT\\System32\\Wbem', u't:\\dev\\python21', u'T:\\dev\\omni\\bin\\x 86_win32', 'C:\\WINNT\\system32', 'C:\\WINNT', 'C:\\WINNT\\S ystem32 \\Wbem', 't:\\dev\\python21', 'T:\\dev\\MSVS\\Common\\Tools\ \ WinNT', 'T:\\dev\\MSVS\\Common\\MSDev98 \\Bin', 'T:\\dev\\MSVS\\Common\\Tools', 'T:\\dev\\MSVS\\VC98 \\bin', 't:\\Tools\\TeX\\bin\ \win32', 'T:\\dev\\omni\\bin\\x86_win32', 'T:\\dev\\root\\bi n', 'T:\\dev\\root\\usr\\X11R6 \\bin', 't:\\dev\\bins', 'T:\\dev\\jdk 1.3.1_01\\bin', 'T:\\dev\\jdk1.3.1_01 \\jre\\bin', 'T:\\dev\\python21 \\PyQt\\bin', 't:\\dev\\python21\\Scripts', 'C:\\Program Fil es\\WinRAR', 'C:\\Program Files\\ARJ', 'T:\\Tools\\WaveMetrics\\Igor Pro Folder', 't:\\Tools\\qt\\bin', 'C:\\Docs\\Boulot\\S\xe9 minaireLORIA-jan2002\\DemoCORBA']

[warsaw]

Logged In: YES user_id=12800

Max M's problem (referred to w/ the above url) appeared to be related to non-ASCII characters in his path. His resolution is here:

http://mail.python.org/pipermail/python-list/2002-January/083957.html

[tim-one]

Logged In: YES user_id=31435

Laurent, the last component of your PATH:

'C:\\Docs\\Boulot\\S\xe9minaireLORIA-jan2002\\DemoCORBA'

in not a Unicode string but contains a "funny character" (chr(0xe9)). It probably doesn't look funny to you \<wink>, but the Unicode subsystem believes by default that non- Unicode strings are plain 7-bit ASCII, and \xe9 isn't legitimate under that belief. Because there are *some* Unicode strings in your PATH, the join attempt needs to convert all strings to Unicode, but it can't do it (the last path component isn't a valid encoding under the default belief).

As a quick workaround, you could remove that entry from your PATH. Or you could set Python's belief about the default encoding to match your belief (it's impossible for Python to guess this, alas -- for all it knows your last path component was intended to be German or Navajo).

[61337411-43fc-4a9c-b8d5-4060aede66d0]

Logged In: YES user_id=21627

I think the right solution would be to apply the file system encoding to each of the strings (either to the Unicode strings, producing byte strings, or vice versa). That, of course, would require that the file system encoding is exposed to the application.

I have a PEP in mind titled "Unicode at system interfaces", which would include exposing the file system encoding, but I wanted to wait to see reactions (hopefully acceptance) of PEP-277 first.

[61337411-43fc-4a9c-b8d5-4060aede66d0]

Logged In: YES user_id=21627

Since this is msvccompiler, it should be safe to encode Unicode strings using the mbcs encoding.

Laurent, can you please try the attached patch and report whether it solves the problem?

[62a07a1b-c248-4c64-9f7a-30815f696946]

Logged In: YES user_id=440351

Ok, loewis patch msvc.diff works well, I does the setup.py build of StandaloneZODB completly.

Now I'm trying StandaloneZODB test.py script (seem to be a long running test).

A+

Laurent.

[61337411-43fc-4a9c-b8d5-4060aede66d0]

Logged In: YES user_id=21627

Committed as msvccompiler.py 1.44.