'Document' object has no attribute 'findall'

[thobryan]

After upgrading ubuntu 13.10 to 14.04. All packages that I am trying to install is getting the following error:

Exception: Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
  status = self.run(options, args)
File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
  requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle) 
File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1177, in prepare_files 
  url = finder.find_requirement(req_to_install, upgrade=self.upgrade) 
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 256, in find_requirement
  page_versions.extend(self._package_versions(page.links, req.name.lower())) 
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 432, in _package_versions 
  for link in self._sort_links(links): 
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 422, in _sort_links 
  for link in links: 
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 769, in links 
  for anchor in self.parsed.findall(".//a"):
AttributeError: 'Document' object has no attribute 'findall'

Storing debug log for failure in /root/.pip/pip.log

[thobryan]

One workaround was to execute "easy_install pip".

[spinza]

Getting the same trying to install a package using pip.

[dstufft]

Can you give the exact command you used?

[spinza]

sudo pip install flexget

[thobryan]

Ironssouw, did you upgrade Ubuntu to 14.04? (In my case I just got this error after upgrade)

[spinza]

Yes.

[dstufft]

What is the value of pip --version.

[spinza]

pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)

[dstufft]

Ok, and the output of pkg -s python-pip ?

[dstufft]

er, dpkg -s python-pip.

[spinza]

$ sudo dpkg -s python-pip
Package: python-pip
Status: install ok installed
Priority: optional
Section: python
Installed-Size: 468
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: all
Version: 1.5.4-1
Depends: python (>= 2.7), python (<< 2.8), python:any (>= 2.7.1-0ubuntu2), ca-certificates, python-colorama, python-distlib, python-html5lib, python-pkg-resources, python-setuptools (>= 0.6c1), python-six, python-requests
Recommends: build-essential, python-dev-all (>= 2.6)
Description: alternative Python package installer
 pip is a replacement for easy_install, and is intended to be an improved
 Python package installer.  It integrates with virtualenv, doesn't do partial
 installs, can save package state for replaying, can install from non-egg
 sources, and can install from version control repositories.
Original-Maintainer: Debian Python Modules Team <python-modules-team@lists.alioth.debian.org>
Homepage: http://www.pip-installer.org/

[spinza]

I removed and reinstalled pip also. Same problem.

[dstufft]

Hmm, I can't reproduce on a fresh 14.04 image... Maybe try uninstalling python-html5lib and reinstalling that?

[spinza]

Still the same. This is an upgrade from 13.10 to 14.04.

[dstufft]

Ok, i'll try that then!

[dstufft]

Nope, still can't reproduce ;(

[Ivoz]

For more output that could help us narrow down the issue:

• What path are you running it under? (could you give a reproduction of the shell commands you issued?)
• Could you try it A) without root, by adding the --user flag instead; and B) try installing inside a virtualenv (python-virtualenv)?
• Also give a paste of ~/.pip/pip.log (from under root's home and yours)?

[schmidtc]

I'm hitting this same problem. Seems to be triggered by installing certain packages. I can reproduce this using a fresh Amazon EC2 instance loaded with "Ubuntu Server 14.04 LTS (PV)". And running the following…

ubuntu:~$ sudo apt-get install python-pip -y
… (clipped) ...
ubuntu:~$ sudo pip install boto
Downloading/unpacking boto
  Downloading boto-2.28.0-py2.py3-none-any.whl (1.2MB): 1.2MB downloaded
Installing collected packages: boto
Successfully installed boto
Cleaning up…
ubuntu:~$ sudo pip uninstall boto -y
Uninstalling boto:
  Successfully uninstalled boto
ubuntu:~$ sudo pip install html5lib==0.90
Downloading/unpacking html5lib==0.90
  Downloading html5lib-0.90.tar.gz (86kB): 86kB downloaded
  Running setup.py (path:/tmp/pip_build_root/html5lib/setup.py) egg_info for package html5lib

Installing collected packages: html5lib
  Found existing installation: html5lib 0.999
    Uninstalling html5lib:
      Successfully uninstalled html5lib
  Running setup.py install for html5lib

Successfully installed html5lib
Cleaning up...
ubuntu:~$ sudo pip install boto
Downloading/unpacking boto
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 256, in find_requirement
    page_versions.extend(self._package_versions(page.links, req.name.lower()))
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 432, in _package_versions
    for link in self._sort_links(links):
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 422, in _sort_links
    for link in links:
  File "/usr/lib/python2.7/dist-packages/pip/index.py", line 769, in links
    for anchor in self.parsed.findall(".//a"):
AttributeError: 'Document' object has no attribute 'findall'

Storing debug log for failure in /home/ubuntu/.pip/pip.log

In summary, apt-get install python-pip. Install and uninstall boto to demonstrate pip is working. Installing html5lib breaks pip.

sudo pip uninstall html5lib seems to fix the problem.

[dstufft]

Ah I see.

So pip itself bundles some software, html5lib being one of those, on various Linux distros they unbundle that software and cause pip to import it normally instead of using it's own bundled copy.

What this means is that if you install an incompatible version of one of pip's dependencies into the global Python, then you'll break pip because pip will import that and attempt to use that instead of the correct one installed by apt-get.

[dstufft]

Going to close this. This is working as expected for the Debian/Ubuntu based systems which debundle things from inside of pip. If you still feel this should be different then the best avenue to take this up with is on the Debian or Ubuntu bug trackers.

[spinza]

I didn't understand the comments about what is wrong. How do I fix it?

On Monday, May 26, 2014, Donald Stufft notifications@github.com wrote:

Ah I see.

So pip itself bundles some software, html5lib being one of those, on various Linux distros they unbundle that software and cause pip to import it normally instead of using it's own bundled copy.

What this means is that if you install an incompatible version of one of pip's dependencies into the global Python, then you'll break pip because pip will import that and attempt to use that instead of the correct one installed by apt-get.

— Reply to this email directly or view it on GitHub https://github.com/pypa/pip/issues/1742#issuecomment-44197785.

__ Louis Rossouw lrossouw@gmail.com | m: +27 82 498 7214

[Ivoz]

@lrossouw try pip installing packages in a virtualenv or using the --user flag instead of sudo installing them as system packages.

The issue seems to be caused by debian's modification of its pip, so we can't address it ourselves.

[cjd]

I hit this as well and "sudo pip uninstall html5lib" fixed it for me

[munirhossain]

Nothing helped for me after I upgraded to Ubuntu 14.04 form Ubuntu 12.04 to fix the problem. Then I completely removed all the pythons from my system and then in installed again and installed pip. Still virtualenv was not working and complaining that virtualenv==1.9.1 not found!! then I found that in the /usr/local/bin/virtualenv file there is strict requirement of 1.9.1 version. Since I removed all the python and related packages and reinstalled the version of virtualenv has been updated, and there is no more virtualenv-1.9.1. So what I did next is that removing the strictly checking of version 1.9.1 in the /usr/local/bin/virtualenv and changed as following. Then it helped. OOH-HOO it took my important 3 hours. Hope people can fix in 3 minutes.

#!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'virtualenv>=1.9.1','console_scripts','virtualenv'
__requires__ = 'virtualenv>=1.9.1'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('virtualenv>=1.9.1', 'console_scripts', 'virtualenv')()
    )

[Ivoz]

@munirhossain it seems that for you the script file /usr/local/bin/virtualenv was left over after uninstallation of python-things, whereas it should have been deleted. Thereafter it took precedence on your path and was continually trying to look for a ghost installation of virtualenv 1.9. If you removed it and reinstalled virtualenv through your preferred method then a proper script should end up on your path, to use normally.

[brycenesbitt]

My machine has no /usr/local/bin/virtualenv

After sudo pip uninstall html5lib I get:

Downloading/unpacking six from https://pypi.python.org/packages/py2.py3/s/six/six-1.8.0-py2.py3-none-any.whl#md5=2f5f96148c68f3c1611f489678a8b445
  Downloading six-1.8.0-py2.py3-none-any.whl
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1259, in prepare_files
    )[0]
IndexError: list index out of range

Storing debug log for failure in /root/.pip/pip.log

[brycenesbitt]

Downstream bug report for Ubuntu 14.04: https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1373254 If you see this also, click "affects me also".

[roldandvg]

Had the same problem and solvent running the following command:

sudo easy_install-U pip

For this you must have previously installed this utility with:

sudo aptitude install python-setuptools

I then proceeded to install the package with pip and all good.

[snehalparmar]

easy_install pip worked for me.

[harrypotterismyname]

@roldandvg approach works for me

[sarepal]

"sudo pip uninstall html5lib" fixed it for me as well

[yonas]

"sudo pip uninstall html5lib" worked for me too.

[kuroyuki]

Seems the issue was fixed in the latest Pip version but apt-get has old one. I installed pip from pypa.io wget https://bootstrap.pypa.io/get-pip.py python get-pip.py And updated to latest version with : sudo pip install --upgrade pip Now I able to install packages w/o problems.