search

collective / mr.bob

Mister Bob (the builder) is filesystem template renderer
http://mrbob.readthedocs.org/en/latest/
Other
69 stars 30 forks source link

Installing mr.bob as requirement raises UnicodeError #70

Closed glenfant closed 9 years ago

glenfant commented 10 years ago

Hi,

I made a package named bobtemplates.gillux that (tries to) install mr.bob as requirement. But when pip installs mr.bob as requirement, it fails with a UnicodeError.

$ pip install bobtemplates.gillux
Downloading/unpacking bobtemplates.gillux
  Downloading bobtemplates.gillux-1.1.0.tar.gz
  Running setup.py (path:/Users/glenfant/.virtualenvs/tmp-c6c81cbdcc2345de/build/bobtemplates.gillux/setup.py) egg_info for package bobtemplates.gillux
    /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'extra_requires'
      warnings.warn(msg)

Requirement already satisfied (use --upgrade to upgrade): setuptools in ./lib/python2.7/site-packages (from bobtemplates.gillux)
Downloading/unpacking mr.bob (from bobtemplates.gillux)
  Downloading mr.bob-0.1.1.zip (43kB): 43kB downloaded
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Users/glenfant/.virtualenvs/tmp-c6c81cbdcc2345de/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Users/glenfant/.virtualenvs/tmp-c6c81cbdcc2345de/lib/python2.7/site-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/glenfant/.virtualenvs/tmp-c6c81cbdcc2345de/lib/python2.7/site-packages/pip/req.py", line 1197, in prepare_files
    do_download,
  File "/Users/glenfant/.virtualenvs/tmp-c6c81cbdcc2345de/lib/python2.7/site-packages/pip/req.py", line 1375, in unpack_url
    self.session,
  File "/Users/glenfant/.virtualenvs/tmp-c6c81cbdcc2345de/lib/python2.7/site-packages/pip/download.py", line 582, in unpack_http_url
    unpack_file(temp_location, location, content_type, link)
  File "/Users/glenfant/.virtualenvs/tmp-c6c81cbdcc2345de/lib/python2.7/site-packages/pip/util.py", line 621, in unpack_file
    unzip_file(filename, location, flatten=not filename.endswith(('.pybundle', '.whl')))
  File "/Users/glenfant/.virtualenvs/tmp-c6c81cbdcc2345de/lib/python2.7/site-packages/pip/util.py", line 499, in unzip_file
    fn = os.path.join(location, fn)
  File "/Users/glenfant/.virtualenvs/tmp-c6c81cbdcc2345de/bin/../lib/python2.7/posixpath.py", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 31: ordinal not in range(128)

Storing debug log for failure in /Users/glenfant/.pip/pip.log

Although my setup.py is "regular" https://github.com/glenfant/bobtemplates.gillux/blob/master/setup.py#L35 When replacing in line 35 "mr.bob" by any other package ("unidecode", "lxml", anything, ...) it works, so I don't think this line is the source of the issue.

The only workaround is to :

pip install mr.bob
pip install bobtemplates.gillux

But I can't say at this stage if the source of that issue is in pip or mr.bob.

domenkozar commented 10 years ago

Looks like a bug in pip, what version are you using?

glenfant commented 10 years ago

My opinion too, but I needed to have yours before filing a ticket to the pip author. I run pip 1.5.8

pfmoore commented 10 years ago

Looks like issue #71. Caused by test files with non-ASCII names triggering a distutils bug (using the local encoding which may not support those characters for the RECORD file)

do3cc commented 9 years ago

This bug has been reported upstream: https://github.com/pypa/pip/issues/1989 It is a pity, because without this issue, I could tell people to install a template in a fresh virtualenv via: pip install git+https://github.com/collective/bobtemplates.plone.git The problem did not exist in mr.bob 0.1.0 so maybe I pin it there to 0.1.0

do3cc commented 9 years ago

Oh, I have been mistaken. Installing mr.bob directly works with any mr.bob version only as a dependency it fails. There is no mr.bob version to which I could pin bobtemplates.plone...

domenkozar commented 9 years ago

That's because we have unicode tests from day 0. Not really sure how to solve this one except from providing a patch upstream and getting it merged

do3cc commented 9 years ago

Maybe removing the test code from the egg release is an option

domenkozar commented 9 years ago

Hopefully this PR is merged soon: https://github.com/pypa/pip/pull/1473

domenkozar commented 9 years ago

@do3cc the other option is to dynamically generate the test file before testing it. I'm willing to accept that patch if someone has the time

idgserpro commented 9 years ago

The same problem happens with zc.buildout if you have mr.bob installed somewhere in python namespace (like pip install --user mr.bob). They are fixing it there, just commenting here if someone is having the same problem.

gforcada commented 9 years ago

Just reporting that this currently works, so maybe it can be closed?

mkdir test
cd test
virtualenv-2.7 .
source bin/activate
pip install bobtemplates.plone
mrbob -O collective.addon bobtemplates:plone_addon

pip version: 6.0.8

pbauer commented 9 years ago

I still have the same issue:

$ virtualenv-2.7 somepy
New python executable in somepy/bin/python2.7
Also creating executable in somepy/bin/python
Installing setuptools, pip...done.
$ cd somepy
$ source bin/activate
(somepy) $ /somepy pip install bobtemplates.plone
Downloading/unpacking bobtemplates.plone
  Downloading bobtemplates.plone-0.11.tar.gz
  Running setup.py (path:/Users/philip/workspace/somepy/build/bobtemplates.plone/setup.py) egg_info for package bobtemplates.plone

    warning: no previously-included files matching '*pyc' found anywhere in distribution
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./lib/python2.7/site-packages (from bobtemplates.plone)
Downloading/unpacking mr.bob (from bobtemplates.plone)
  Downloading mr.bob-0.1.1.zip (43kB): 43kB downloaded
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Users/philip/workspace/somepy/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Users/philip/workspace/somepy/lib/python2.7/site-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/philip/workspace/somepy/lib/python2.7/site-packages/pip/req.py", line 1197, in prepare_files
    do_download,
  File "/Users/philip/workspace/somepy/lib/python2.7/site-packages/pip/req.py", line 1375, in unpack_url
    self.session,
  File "/Users/philip/workspace/somepy/lib/python2.7/site-packages/pip/download.py", line 582, in unpack_http_url
    unpack_file(temp_location, location, content_type, link)
  File "/Users/philip/workspace/somepy/lib/python2.7/site-packages/pip/util.py", line 621, in unpack_file
    unzip_file(filename, location, flatten=not filename.endswith(('.pybundle', '.whl')))
  File "/Users/philip/workspace/somepy/lib/python2.7/site-packages/pip/util.py", line 499, in unzip_file
    fn = os.path.join(location, fn)
  File "/Users/philip/workspace/somepy/lib/python2.7/posixpath.py", line 73, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 31: ordinal not in range(128)

Storing debug log for failure in /Users/philip/.pip/pip.log
gforcada commented 9 years ago

@pbauer which pip version are you using?

pbauer commented 9 years ago

@gforcada pip 1.5.6 (it comes when building a fresh python2.7 using https://github.com/collective/buildout.python)

gforcada commented 9 years ago

@pbauer according to pypi that's quite old: https://pypi.python.org/pypi/pip

do3cc commented 9 years ago

It might very well be that this is the pip version you got when you installed buildout.python. Maybe a fresh buildout.python has a newer version. (Probably)

pbauer commented 9 years ago

No, my buildout.python is 3 weeks old. But after updating pip with pip install -U pip the installation with pip install bobtemplates.plone worked like a charm. So we should add to the docs that pip >= 6.0 is required for that (that is the lowest version of pip where the insatllation worked).

domenkozar commented 9 years ago

If someone has time to write a patch that creates the test file at runtime instead of having the file in the distribution, I'm willing to change that and release a minor fix.

domenkozar commented 9 years ago

I've released 0.1.2 that ships with tarball instead of zipfile and that should fix installation+unicode problems. Reopen if the problem appears again

mauritsvanrees commented 9 years ago

A pip install of bobtemplates.plone with mr.bob as dependency works, as long as you use a recent pip (7.1.2 here).

Uninstall fails though:

mauritsvanrees@procyon:venv4 $ . bin/activate
(venv4)mauritsvanrees@procyon:venv4 $ pip install -U pip
Downloading/unpacking pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl#md5=5ff9fec0be479e4e36df467556deed4d
  Downloading pip-7.1.2-py2.py3-none-any.whl (1.1MB): 1.1MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.5.6
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...
(venv4)mauritsvanrees@procyon:venv4 $ pip install mr.bob
Collecting mr.bob
  Using cached mr.bob-0.1.2.tar.gz
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./lib/python2.7/site-packages (from mr.bob)
Collecting six>=1.2.0 (from mr.bob)
  Using cached six-1.9.0-py2.py3-none-any.whl
Collecting Jinja2>=2.5.0 (from mr.bob)
  Using cached Jinja2-2.8-py2.py3-none-any.whl
Collecting MarkupSafe (from Jinja2>=2.5.0->mr.bob)
  Using cached MarkupSafe-0.23.tar.gz
Installing collected packages: six, MarkupSafe, Jinja2, mr.bob
  Running setup.py install for MarkupSafe
  Running setup.py install for mr.bob
Successfully installed Jinja2-2.8 MarkupSafe-0.23 mr.bob-0.1.2 six-1.9.0
(venv4)mauritsvanrees@procyon:venv4 $ pip uninstall mr.bob
Uninstalling mr.bob-0.1.2:
  /Users/mauritsvanrees/tmp/venv4/bin/mrbob
...
Proceed (y/n)? y
Exception:
Traceback (most recent call last):
  File "/Users/mauritsvanrees/tmp/venv4/lib/python2.7/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/Users/mauritsvanrees/tmp/venv4/lib/python2.7/site-packages/pip/commands/uninstall.py", line 76, in run
    requirement_set.uninstall(auto_confirm=options.yes)
  File "/Users/mauritsvanrees/tmp/venv4/lib/python2.7/site-packages/pip/req/req_set.py", line 305, in uninstall
    req.uninstall(auto_confirm=auto_confirm)
  File "/Users/mauritsvanrees/tmp/venv4/lib/python2.7/site-packages/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Users/mauritsvanrees/tmp/venv4/lib/python2.7/site-packages/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/Users/mauritsvanrees/tmp/venv4/lib/python2.7/site-packages/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/Users/mauritsvanrees/buildout/python2.7/parts/opt/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/Users/mauritsvanrees/buildout/python2.7/parts/opt/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/Users/mauritsvanrees/buildout/python2.7/parts/opt/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/Users/mauritsvanrees/tmp/venv4/lib/python2.7/site-packages/mrbob/tests/templates/encoding\xc4\x8d/map\xc4\x8da/\xc4\x87a.bob'

See also https://github.com/plone/bobtemplates.plone/issues/107

domenkozar commented 9 years ago

Yeah, it unfortunately requires a quite fresh version of pip. Not sure if that dependency should be claimed.