Cannot download extensions

[mcdonc]

After running bootstrap in a fresh checkout of buildout.python as of today:

$ bin/buildout
Getting distribution for 'buildout.extensionscripts'.
While:
  Installing.
  Loading extensions.
  Getting distribution for 'buildout.extensionscripts'.
Error: Couldn't find a distribution for 'buildout.extensionscripts'.

It's indeed in the PyPI index. I tried changing zc.buildout.easy_install from:

default_index_url = os.environ.get(
    'buildout-testing-index-url',
    'http://pypi.python.org/simple',
    )

to

default_index_url = os.environ.get(
    'buildout-testing-index-url',
    'https://pypi.org/simple/',
    )

But no dice. I'm gonna guess this is fixed in later 2.X versions of buildout in combination with "new" PyPI but....

[mpeeters]

I have the same issue on my new Mac with OS X Mojave and it seems related to OS X : https://travis-ci.org/collective/buildout.python/jobs/472114783

[daggelpop]

Hello, I have the same problem on Ubuntu 18.04 (Python 2.7.15rc1).

[mpeeters]

I will have a look on this problem ASAP

[mpeeters]

I fix this problem on OS X Mojave by installing headers : open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

[mjpieters]

That sounds like an unlikely fix; the SDK headers won't help buildout load packages from PyPI.

[mjpieters]

I've added a GitHub Actions workflow, and bootstrapping actually works on Ubuntu 16.04. Building 2.4 and 3.2 fails as those versions can't cope with the newer OpenSSL versions available, but it's a step further than we were before.

Bootstrapping on Ubuntu 18.04 and MacOS, not so much. I'll see what I can do to improve this.

[mjpieters]

I haven't got a solution yet, but I did find the reason why this fails: zc.buildout.easy_install.Installer._obtain() uses () as a sentinel to compare with pkg_resources._vendor.packaging.version.Version() instances, and this comparison is always false for current setuptools versions.

Hence, there is never a best version, and the install fails. On platforms where things happen to work (the Github Actions Ubuntu 16.04 env for example) use an older setuptools where the comparison does work (with the tuple sorting before the distribution version), either because the system Python doesn't have setuptools installed (and so bootstrap installs setuptools==1.4.2) or the system Python setuptools version is still ancient enough.

[mjpieters]

In fact, the 16.04 builds actually tell us what is happening:

/usr/lib/python2.7/dist-packages/pkgresources/__init_\.py:187: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.

This warning was created just for buildout, see the 8.01 setuptools release notes:

• Issue #296: Restore support for iteration over parse_version result, but deprecated that usage with a warning. Fixes failure with buildout.

The deprecation was finally enacted in version 39.0. Using a virtualenv Python with pip install -f 'setuptools<39' indeed makes things work again.

[chrismattmann]

interesting so if I use the old setuptools like https://github.com/collective/buildout.python/commit/a721d1e7306a6fc6c6f57e5c9de15e77dc46b262 that's a possible workaround?

[mjpieters]

Resolved in 707b60cd9d4f0ce6844392a7b555340c89011098

[mjpieters]

interesting so if I use the old setuptools like a721d1e that's a possible workaround?

It was only a stopgap measure; the real fix was to install a local egg of setuptools at bootstrap time. That way only this project is stuck with ageing versions, not your other Python projects.

[chrismattmann]

awesome @mjpieters i will try it out today!