Installation: python bootstrap.py fails

[harsham05]

Hi @chrismattmann

I browsed through past issues on ETL lib repo but the installation process seems to have changed. I have upgraded & verified setuptools 18.4

harsha@ubuntu:~/Documents/CS572DR/etllib$ python bootstrap.py Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-18.4.zip Traceback (most recent call last): File "bootstrap.py", line 92, in ez'use_setuptools' File "", line 170, in use_setuptools File "", line 121, in _do_download File "", line 332, in download_setuptools File "", line 252, in download_file_curl File "", line 207, in _clean_check File "/home/harsha/anaconda/lib/python2.7/subprocess.py", line 540, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['curl', 'https://pypi.python.org/packages/source/s/setuptools/setuptools-18.4.zip', '--silent', '--output', '/tmp/tmpA9DKxF/setuptools-18.4.zip']' returned non-zero exit status 77

Thank you very much, Harsha

[chrismattmann]

try python -d bootstrap.py

[harsham05]

Thank you @chrismattmann , but I still get the same error with python -d bootstrap.py

[harsham05]

@chrismattmann ,

Since I had installed setuptools manually, I commented out lines 68-92 in bootstrap.py

######################################################################
# load/install setuptools
''' 
try:
    if options.allow_site_packages:
        import setuptools
        import pkg_resources
    from urllib.request import urlopen
except ImportError:
    from urllib2 import urlopen
ez = {}
exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez)
if not options.allow_site_packages:
    # ez_setup imports site, which adds site packages
    # this will remove them from the path to ensure that incompatible versions 
    # of setuptools are not in the path
    import site
    # inside a virtualenv, there is no 'getsitepackages'. 
    # We can't remove these reliably
    if hasattr(site, 'getsitepackages'):
        for sitepackage_path in site.getsitepackages():
            sys.path[:] = [x for x in sys.path if sitepackage_path not in x]
setup_args = dict(to_dir=tmpeggs, download_delay=0)
ez['use_setuptools'](**setup_args)
'''

[harsham05]

@chrismattmann , UPDATE: Looking at https://github.com/chrismattmann/etllib/issues/36#issuecomment-72591659 Dongni got a different runtime warning. Please let me know if the lines I commented out are fine since sys.path(from line 89) is iterated through in line 98

_If so, I propose adding manual installation of setuptools as a dependency to ETL lib in the README (since setuptools installation/upgrade is finicky)_

harsha@ubuntu:~/Documents/CS572DR/etllib$ python -d bootstrap.py /home/harsha/anaconda/lib/python2.7/site-packages/setuptools-18.4-py2.7.egg/pkg_resources/init.py:203: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need. Creating directory '/home/harsha/Documents/CS572DR/etllib/eggs'. Creating directory '/home/harsha/Documents/CS572DR/etllib/bin'. Creating directory '/home/harsha/Documents/CS572DR/etllib/parts'. Creating directory '/home/harsha/Documents/CS572DR/etllib/develop-eggs'. Generated script '/home/harsha/Documents/CS572DR/etllib/bin/buildout'.

[chrismattmann]

We need to pin the setup tools that will fix it I reported it and they fixed in later version let me look it up

[harsham05]

@chrismattmann , I was able to manually install latest setuptools (18.4) using below commands. I will try updating the same in bootstrap.py

wget --no-check-certificate https://bootstrap.pypa.io/ez_setup.py
python ez_setup.py --insecure

[chrismattmann]

See this issue: https://github.com/collective/buildout.python/pull/48 looks like the latest buildout fixes it.

[chrismattmann]

@harsham05 did that fix it?

[harsham05]

@chrismattmann I tried to pin the following in base.cfg and extended the same file in buildout.cfg

[versions]
setuptools = 1.4.2

But python -d bootstrap.py still breaks trying to fetch setuptools-18.4

[chrismattmann]

hrm. I just installed this without error on a new CentOS box. What env are you installing on?

[chrismattmann]

also it seems like when you tried this the URL for setuptools-18.4.zip was broke (I wonder if you were offline but just didn't know it?)

Can you blow away your ETLlib install and try starting from scratch?

[harsham05]

@chrismattmann Ubuntu 14.04.3 LTS

Prof, I started with a clean install just now, may I please know where you have pinned following?

[versions]
setuptools = 1.4.2

[chrismattmann]

I haven't. Try a clean install with no changes. Worked fine for me on CentOS 7.

[harsham05]

@chrismattmann

Hi Prof, I was able to successfully install on my CentOS 6.4 Virtual Machine with Anaconda Python2.7 :)

Thank you

python -d bootstrap.py 
Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-18.4.zip
Extracting in /tmp/tmpF1knPU
Now working in /tmp/tmpF1knPU/setuptools-18.4
Building a Setuptools egg in /tmp/tmpS7ukxQ
/tmp/tmpS7ukxQ/setuptools-18.4-py2.7.egg
/tmp/tmpS7ukxQ/setuptools-18.4-py2.7.egg/pkg_resources/__init__.py:203: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.
/home/harsha/anaconda/lib/python2.7/site-packages/setuptools-14.3-py2.7.egg/pkg_resources/__init__.py:2512: PEP440Warning: 'llvmlite (0.2.2-1-gbcb15be)' is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
Creating directory '/home/harsha/dr/etllib/eggs'.
Creating directory '/home/harsha/dr/etllib/bin'.
Creating directory '/home/harsha/dr/etllib/parts'.
Creating directory '/home/harsha/dr/etllib/develop-eggs'.
Generated script '/home/harsha/dr/etllib/bin/buildout'.

bin/buildout install with-tika
Develop: '/home/harsha/dr/etllib/.'
Getting distribution for 'zc.recipe.egg>=2.0.0a3'.
Got zc.recipe.egg 2.0.3.
Installing with-tika.
Getting distribution for 'tika>=1.8.7'.
Got tika 1.11.
Getting distribution for 'python-magic'.
Got python-magic 0.4.10.
Getting distribution for 'iso8601'.
zip_safe flag not set; analyzing archive contents...
Got iso8601 0.1.10.
Getting distribution for 'requests'.
Got requests 2.8.1.
Generated script '/home/harsha/dr/etllib/bin/poster'.
Generated script '/home/harsha/dr/etllib/bin/translatejson'.
Generated script '/home/harsha/dr/etllib/bin/repackageandpost'.
Generated script '/home/harsha/dr/etllib/bin/repackage'.
Generated script '/home/harsha/dr/etllib/bin/imagesimilarity'.
Generated script '/home/harsha/dr/etllib/bin/tsvtojson'.

[chrismattmann]

Yay! Can you close this