Installation process fail on cleanup [French character]

Hi,

The installation process fail on cleanup because of accentuated characters in error messages (French messages).

The stack trace:

[...]
Successfully installed cookiecutter click jinja2 binaryornot mock PyYAML markupsafe
Cleaning up...
  Exception:
Traceback (most recent call last):
  File "D:\MyName\Projets\virtualenv\py-cook\lib\site-packages\pip\basecommand.py", line 122, in main
    status = self.run(options, args)
  File "D:\MyName\Projets\virtualenv\py-cook\lib\site-packages\pip\commands\install.py", line 302, in run
    requirement_set.cleanup_files(bundle=self.bundle)
  File "D:\MyName\Projets\virtualenv\py-cook\lib\site-packages\pip\req.py", line 1333, in cleanup_files
    rmtree(dir)
  File "D:\MyName\Projets\virtualenv\py-cook\lib\site-packages\pip\util.py", line 43, in rmtree
    onerror=rmtree_errorhandler)
  File "C:\Python27\Lib\shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\Lib\shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\Lib\shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Python27\Lib\shutil.py", line 256, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Python27\Lib\shutil.py", line 254, in rmtree
    os.rmdir(path)
WindowsError: [Error 145] Le rÚpertoire nÆest pas vide: 'D:\\MyName\\Projets\\virtualenv\\py-cook\\build\\PyYAML\\tests\\lib3'

Traceback (most recent call last):
  File "C:\Python27\Lib\runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python27\Lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "D:\MyName\Projets\virtualenv\py-cook\Scripts\pip.exe\__main__.py", line 9, in <module>
  File "D:\MyName\Projets\virtualenv\py-cook\lib\site-packages\pip\__init__.py", line 185, in main
    return command.main(cmd_args)
  File "D:\MyName\Projets\virtualenv\py-cook\lib\site-packages\pip\basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 1108: ordinal not in range(128)

How to reproduce:

Use a French distribution of Windows 7 ;-)
Install Python 2.7.x
Install setuptools and virtualenv libraries
Install Microsoft compiler for Python (http://www.microsoft.com/en-us/download/details.aspx?id=44266)
Install Git
Create a virtualenv
Install cookiecutter with pip install cookiecutter.

Everything should go fine except the cleanup…

If you retry pip install cookiecutter, it says:

Requirement already satisfied (use --upgrade to upgrade): cookiecutter in d:\MyName\projets\virtualenv\py-cook\lib\site-packages
Cleaning up...

Diagnostic:

This is a classic error because MS-DOS in French use the codepage cp850. So WindowsError contains a byte string message with non-ASCII characters (like the "é" in "Le répertoire n’est pas vide"– "The directory is not empty").

How to fix that:

Unfortunatly, sys.getdefaultencoding() doesn't work, it gives: ascii which is the wrong answer. You should use sys.stdin.encoding (if it is defined):

>python -c "import sys; print(sys.stdin.encoding)"
cp850

That way, you can convert the byte string in unicode string using the cp850 encoding, then convert accentuated characters in their non-accentuated equivalent.

Here the convert function:

import unicodedata

def normalize(str_):
    ''' Normalize given string to an ASCII string
    (convert accents to non accented characters, ...)

    :param str_: the string to normalize
    '''
    return unicodedata.normalize('NFKD', unicode(str_)).encode('ascii', 'ignore')

Hope it helps. – Laurent.

audreyfeldroy / cookiecutter-pypackage

Installation process fail on cleanup [French character] #56