Closed idgserpro closed 8 years ago
The same, breaks Vagrant/Puppet installation:
wget https://bootstrap.pypa.io/ez_setup.py -O /.puphpet-stuff/ez_setup.py && python /.puphpet-stuff/ez_setup.py
--2016-04-25 15:46:44-- https://bootstrap.pypa.io/ez_setup.py
Resolving bootstrap.pypa.io (bootstrap.pypa.io)... 185.31.17.175
Connecting to bootstrap.pypa.io (bootstrap.pypa.io)|185.31.17.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12059 (12K) [text/x-python]
Saving to: ‘/.puphpet-stuff/ez_setup.py’
100%[====================================================================================================================================================================================================>] 12,059 --.-K/s in 0s
2016-04-25 15:46:44 (77.5 MB/s) - ‘/.puphpet-stuff/ez_setup.py’ saved [12059/12059]
Extracting in /tmp/tmp32iyZh
Traceback (most recent call last):
File "/.puphpet-stuff/ez_setup.py", line 415, in <module>
sys.exit(main())
File "/.puphpet-stuff/ez_setup.py", line 412, in main
return _install(archive, _build_install_args(options))
File "/.puphpet-stuff/ez_setup.py", line 52, in _install
with archive_context(archive_filename):
File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/.puphpet-stuff/ez_setup.py", line 103, in archive_context
with ContextualZipFile(filename) as archive:
File "/.puphpet-stuff/ez_setup.py", line 87, in __new__
return zipfile.ZipFile(*args, **kwargs)
File "/usr/lib/python2.7/zipfile.py", line 770, in __init__
self._RealGetContents()
File "/usr/lib/python2.7/zipfile.py", line 811, in _RealGetContents
raise BadZipfile, "File is not a zip file"
zipfile.BadZipfile: File is not a zip file
I just hit the same "File is not a zip file" error by simply downloading ez_setup.py from the official recommended source and running it
Seems that the problem is: https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package and ez_setup has the old pypi URL format hard-coded.... (He use https://pypi.python.org/pypi/setuptools/json to get last version but not right URLs :-( )
@jaraco Do you have any idea if it's possible to add a test case in setuptools for this situation?
@cyrilgdn Indeed, the xml invalid response from the url comes as a 313 bytes zip file. Try https://pypi.python.org/packages/source/s/setuptools/setuptools-20.10.1.zip and you get
<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>source/s/setuptools/setuptools-20.10.1.zip</Key>
<RequestId>DAC65125FC40ADDB</RequestId>
<HostId>NqleAbKasjoaBoOZhowtOhluWuTzEgiZZ0HIbREPxqq9wxn6VbbdT0P5r+okwXAk4lzIq597yFA=</HostId>
</Error>
https://pypi.python.org/packages/source/s/setuptools/setuptools-20.9.0.zip works.
@dstufft if possible, any news for https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package#comment-27245473?
@jaraco would it be the case to add a _resolve_setuptools_url_download
like we have _resolve_version
in ez_setup.py?
There's a pull request if someone would review it please.
+1
Can we safely use pypi.io (see the full pull request from @jimfulton https://github.com/pypa/setuptools/issues/559) in the state it's now, in pre-production? As of now, we have these statistics for setuptools:
Downloads (All Versions):
581 downloads in the last day
1173985 downloads in the last week
5004041 downloads in the last month
Is pypi.io able to handle the load? I'm not into PYPI 2.0 project so I don't have clues about that.
IMHO, I would like to have the fix done for pypi.io in https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package#comment-27243225 in pypi.python.org as well, but don't know if it's possible. @dstufft is it possible/feasible? Is this workaround temporary? If so, for how long?
Folks, my PR fixes this.
It used pypi.io and it adds the --location option to curl.
Can you post the full output of curl -I -L https://pypi.io/packages/source/s/setuptools/setuptools-20.10.1.zip
?
I will try to merge the PR and update the bootstrap bookmark.
@AltxSF Sorry, can you do that one more time with a -v
added to it as well?
@AltxSF Try it now and see if it works.
Ok good, misconfiguration in the CDN. Sorry about that!
@dstufft I might be missing something. What exactly do I need to bypass the error. I ran curl https://bootstrap.pypa.io/ez_setup.py -o - | python and same error persists
@ewang320 try curl https://bootstrap.pypa.io/ez_setup.py -o - | python - --version=20.9.0
I've merged the pull requests (#559 and #560) for the bootstrap script. Please report back if that doesn't correct the issue.
Sorry, I'm still getting the same error. Downloaded ez_setup from here, but still get:
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 807, in _RealGetContents
raise BadZipfile, "File is not a zip file"
zipfile.BadZipfile: File is not a zip file
Is that the full traceback?
Do you have curl or wget installed?
Sorry. Full output is here:
%> python ez_setup.py
Extracting in /var/folders/5f/zvqgl88j47zbqzcydcncpzz8nvzfxy/T/tmpQjnOme
Traceback (most recent call last):
File "ez_setup.py", line 415, in <module>
sys.exit(main())
File "ez_setup.py", line 412, in main
return _install(archive, _build_install_args(options))
File "ez_setup.py", line 52, in _install
with archive_context(archive_filename):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "ez_setup.py", line 103, in archive_context
with ContextualZipFile(filename) as archive:
File "ez_setup.py", line 87, in __new__
return zipfile.ZipFile(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 766, in __init__
self._RealGetContents()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 807, in _RealGetContents
raise BadZipfile, "File is not a zip file"
zipfile.BadZipfile: File is not a zip file
Also, I have used the suggested curl to get v.20.9 so I'm moving on in the meanwhile, but will try and get 20.10 asap. Thanks
@FrancescoRizzi, I think the issue might be that you have a bad download locally. If ez_setup.py finds a local copy, it doesn't attempt to download it again. If you remove the local (probably small) zip file and re-run the script, I expect it will work. I've tested the latest bootstrap script on OS X and Ubuntu (with and without curl) and the download executes normally as expected.
In #562, I also confirmed the fix works for Windows.
@jaraco When we use bootstrap-buildout.py, and there's a bad download locally, it doesn't matter because setuptools is always downloaded: if you want to use a local copy, you need to explicitly call
python bootstrap-buildout.py --setuptools-to-dir=.
Shouldn't we have the same behavior in ez_setup.py? Always download but give the option to use a local copy, if needed? We're asking this because this "bad download" error is really confusing for beginners, since this is the official primary method of installation.
Anyways, thanks for the fast fix.
Shouldn't we have the same behavior in ez_setup.py?
@idgserpro: I agree. Would you ticket it and consider making a PR?
@jaraco thanks for the info. Where would ez_setup find a bad local copy "locally"? I'm on a Mac OS X El Capitan (10.11.4), and ez_setup was running in an empty directory just created as virtualenv (specifically via virtualenvwrapper mkproject).
I tried to create a new virtualenv, and it seems to install setuptools-18.2.dist-info right away (at least lssitepackages shows that). For added fun, if I'm out of any virtualenv and do a pip list, I find setuptools (20.3.1)
@FrancescoRizzi Alongside ez_setup.py itself. And in this line, if there's already a local file, it doesn't download it again.
We tested with vanilla python ez_setup.py
, don't know exactly what could be different from a virtualenv like yourself.
@jaraco Done. Check https://github.com/pypa/setuptools/issues/563. We we'll try to do a PR.
My suspicion is that @FrancescoRizzi 's ez_setup is deciding not to use wget or curl and is falling back to the "insecure" code and that that code isn't handling the redirect properly.
I'm going to play with this later when I have some time. It might though be helpful for @FrancescoRizzi to add a pdb.sey_trace() to ez_setup.py to walk through what it's actually doing.
@jaraco I didn't follow setuptools bitbucket-github migration, but this issue was created 5 days ago in bitbucket https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package. Are you planning to keep both repositories? Since you migrated the issues to github as well, wouldn't it be the case to at least close bitbucket issues? Just my 0.02c.
@idgserpro: That issue was created against the PyPI project, not Setuptools. A distinction which I missed initially as well. Setuptools issues are marked private, so aren't visible except to developers.
Hi.. I gave it another twirl this morning. Sadly I'm still getting the Bad Zip Error.
I tried it in 3 separate virtualenvs:
1) a virtualenv that existed and had setuptools 18.2, but from which I removed everything that seemed related to setuptools and/or easy_install (both from the project and virtualenv/lib/python2.7/site-packages
2) a virtualenv that existed (had setuptools 18.2), where I didn't do the steps I did in 1
3) a new virtualenv, just created
In all 3 cases I grabbed ez_setup.py this morning, and ran python with it.
The stack trace from test3 (below) seems to indicate download_setuptools elected to actually download a new copy of setuptools (20.10.1)
Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-20.10.1.zip
Extracting in /var/folders/5f/zvqgl88j47zbqzcydcncpzz8nvzfxy/T/tmpJiCjcG
Traceback (most recent call last):
File "ez_setup.py", line 415, in <module>
sys.exit(main())
File "ez_setup.py", line 412, in main
return _install(archive, _build_install_args(options))
File "ez_setup.py", line 52, in _install
with archive_context(archive_filename):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "ez_setup.py", line 103, in archive_context
with ContextualZipFile(filename) as archive:
File "ez_setup.py", line 87, in __new__
return zipfile.ZipFile(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 766, in __init__
self._RealGetContents()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 807, in _RealGetContents
raise BadZipfile, "File is not a zip file"
zipfile.BadZipfile: File is not a zip file
In fact, at the end of this run, there is a (newly created/downloaded) setuptools-20.10.1.zip next to ez_setup.py, sized at 313 bytes. I opened it via double-click (on Mac) to peek, and found just a setuptools-20.10.1.zip.cpgz inside.
Am willing to add print statements (or pdb) to ez_setup and rerun, if we're looking for anything in particular.
@FrancescoRizzi maybe your network is making cache:
@idgserpro you got it!
So... a hard reload/refresh of the ez_setup source got me a new copy (which i recognized as such thanks to the added Maintained at https://github.com/pypa/setuptools/tree/bootstrap.
comment at the top). Running that ez_setup seems to finally get me an acceptable setuptools-20.10.1
If I may suggest, I'd add a version number at the top of ez_setup (it may not change often, but when it does it may help in recognizing a specific version of the file).
Thanks though! Now off to hunt every copy of the old ez_setup I have locally :D
If I may suggest, I'd add a version number at the top of ez_setup (it may not change often, but when it does it may help in recognizing a specific version of the file).
Good idea.
@jaraco buildout does this with bootstrap.py.
Why not just use the date of the file, rather than add additional steps to the change process (and potential for mismatch)?
What do you mean by "date of the file"?
Is ez_setup.py synced every 15 minutes to https://bootstrap.pypa.io/ez_setup.py like @dstufft suggested in https://github.com/buildout/buildout/issues/86#issuecomment-114944317?
What do you mean by "date of the file"?
$ curl -s -I https://bootstrap.pypa.io/ez_setup.py | grep Last-Modified Last-Modified: Tue, 26 Apr 2016 16:30:27 GMT
Annoyingly, Github doesn't seem to provide a Last-Modified date, but does provide an ETag header:
$ curl -s -I https://raw.githubusercontent.com/pypa/setuptools/bootstrap/ez_setup.py | grep ETag ETag: "e05e52bf1d9cdeed93c16f6456e2c532f343fa22"
Is ez_setup.py synced every 15 minutes to ...
Yes.
How to test:
Download
https://bootstrap.pypa.io/bootstrap-buildout.py
At the time of this ticket,
__version__
of this file is 2015-07-01.Run
python bootstrap-buildout.py
You get
You get the same error with 20.10.0.
python bootstrap-buildout.py --setuptools-version=20.10.0
20.9.0 works fine, so this bug was introduced in 20.10.x.
python bootstrap-buildout.py --setuptools-version=20.9.0