pip upgrade to 1.9.1 from 1.9.0

[jrherr]

I'm trying to update our Ubuntu server installation of 1.9.0 to 1.9.1 and banging my head with the following error:

Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 223, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 280, in run
    requirement_set.prepare_files(finder)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 317, in prepare_files
    functools.partial(self._prepare_file, finder))
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 304, in _walk_req_to_install
    more_reqs = handler(req_to_install)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 469, in _prepare_file
    session=self.session)
  File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 825, in unpack_url
    session,
  File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 673, in unpack_http_url
    from_path, content_type = _download_http_url(link, session, temp_dir)
  File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 886, in _download_http_url
    _download_url(resp, link, content_file)
  File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 621, in _download_url
    for chunk in progress_indicator(resp_read(4096), 4096):
  File "/usr/local/lib/python2.7/dist-packages/pip/utils/ui.py", line 133, in iter
    for x in it:
  File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 586, in resp_read
    decode_content=False):
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/response.py", line 307, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/response.py", line 243, in read
    data = self._fp.read(amt)
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/cachecontrol/filewrapper.py", line 54, in read
    self.__callback(self.__buf.getvalue())
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/cachecontrol/controller.py", line 224, in cache_response
    self.serializer.dumps(request, response, body=body),
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/cachecontrol/serialize.py", line 81, in dumps
    ).encode("utf8"),
MemoryError

I'm using pip (sudo -H pip install qiime and many other derivations on that) -- everything else should be up to date. Is this a utf8 issue?

[gregcaporaso]

@jrherr, can you give some information on the system you're installing on? What version of Ubuntu? What is the output of running sudo pip freeze.

These instructions should be really helpful for getting QIIME 1.9.1 installed on Ubuntu.

[jrherr]

I based the installation on the StarCluster base x86_64 AMI, cat /etc/lsb-release gives:

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.5 LTS"

...and sudo pip freeze gives:

backports.ssl-match-hostname==3.4.0.2
BamM==1.4.1
biom-format==2.1.3
burrito==0.9.0
burrito-fillings==0.1.0
bz2file==0.98
certifi==14.5.14
chardet==2.0.1
Cheetah==2.4.4
cogent==1.5.3
configobj==4.7.2
Cython==0.21.2
docutils==0.12
drmaa==0.5
dumbo==0.21.35
emperor==0.9.51
future==0.14.3
gdata==2.0.18
GnuPGInterface==0.3.2
h5py==2.4.0
ipython==2.4.1
Jinja2==2.7.3
khmer==1.3
lazr.uri==1.0.3
M2Crypto==0.21.1
MarkupSafe==0.23
matplotlib==1.4.3
mock==1.0.1
mpi4py==1.2.2
natsort==3.5.2
nose==1.3.4
numexpr==2.4
numpy==1.9.1
numpydoc==0.5
oauth==1.0.1
PAM==0.4.2
pandas==0.15.2
pycurl==7.19.0
pyfasta==0.5.2
Pygments==2.0.2
pynast==1.2.2
pyparsing==2.0.3
pyqi==0.3.2
pyserial==2.5
python-apt===0.8.3ubuntu7.3
python-dateutil==2.4.0
python-debian===0.1.21ubuntu1
pytz==2014.10
pyzmq==14.5.0
qcli==0.1.1
qiime-default-reference==0.1.1
scikit-bio==0.2.3
scikits.statsmodels==0.3.1
scipy==0.15.1
screed==0.8
simplejson==2.3.2
six==1.9.0
Sphinx==1.2.3
tables==2.3.1
tornado==4.0.2
Twisted-Core==11.1.0
Twisted-Names==11.1.0
Twisted-Web==11.1.0
typedbytes==0.3.8
ufw==0.31.1.post1
urwid==1.0.1
virtualenv==1.7.1.2
wadllib==1.3.0
xlrd==0.6.1
zope.interface==3.6.1

I'm now allocating more memory so hopefully that will fix the issue... I'll work on it and get back to you on the fix. Thanks for the link and the input! Greatly appreciated!

[gregcaporaso]

Can you also run print_qiime_config.py and post the output? You mentioned that this is an upgrade so I want to see what's on there already.

[jrherr]

I wasn't able to print_qiime_config.py because during the installation everything broke.

I googled around and this solved my problem. Also, this post helped. Basically I had to:

1. fix the python-dev install on ubuntu, then
2. uninstall a bunch QIIME dependencies with pip, and then
3. reinstall with sudo pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs pip install -U, pausing to fix any packages (by uninstalling them from a residual previous attempt).

Everything is working fine now... Running some QIIME scripts to test the upgrade. Thanks for all your help @gregcaporaso!

[gregcaporaso]

Glad to hear it's working now!