Support setuptools > 38.7.0

[Rotonen]

Setuptools 39.0.0 deprecated some old ways of dealing with parsed versions.

https://github.com/pypa/setuptools/blob/master/CHANGES.rst#v3900

This was recently fixed in buildout.

https://github.com/buildout/buildout/issues/442

But it seems this package is the next one on the firing line for at least a Plone 4.3 deployment on the 4.3 version of this package.

Installing instance.
While:
  Installing instance.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/Users/<redacted>/.homebrew/var/pyenv/versions/2.7.14/envs/<redacted>/lib/python2.7/site-packages/zc/buildout/buildout.py", line 2127, in main
    getattr(buildout, command)(args)
  File "/Users/<redacted>/.homebrew/var/pyenv/versions/2.7.14/envs/<redacted>/lib/python2.7/site-packages/zc/buildout/buildout.py", line 797, in install
    installed_files = self[part]._call(recipe.install)
  File "/Users/<redacted>/.homebrew/var/pyenv/versions/2.7.14/envs/<redacted>/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1557, in _call
    return f()
  File "/Users/<redacted>/.buildout/eggs/cp27m/plone.recipe.zope2instance-4.3-py2-none-any.ovo/plone/recipe/zope2instance/__init__.py", line 95, in install
    major, minor = parsed[0:2]
TypeError: 'Version' object has no attribute '__getitem__'

[idgserpro]

I can confirm this error. I just had to pin setuptools to 38.7.0.

[ida]

Can confirm the error, but pinning doesn't resolve it.

Steps to reproduce:

• Installed virtualenv with Python-2.7.14
• Create a virtualenv
• Upgrade pip of the virtenv
• Install newest buildout with pip of virtenv
• Run buildout with the config minimalplone4 provides, only adding the eggs-directory-option to point to my eggs-cache.

Patched that by adding this line in plone/recipe/zope2instance/__init__.py:

            parsed = parsed.base_version.split('.')

Right under the line where the variable parsed is assigned for the first time (in vs. 4.2.22 that is line 94, tested with various versions).

Suggesting to resolve the error with a try- and except-block, including the line above, instead of working around with a pin or hassling with setuptools, which looks like the root of the problem.

If there are no objections, I can provide a PR these days.

[ida]

Cannot edit the comment above for unknown reasons. Just adding that I pinned setuptools to the adviced version in the minimal-config also, of course.

[idgserpro]

Can confirm the error, but pinning doesn't resolve it.

Pinning and using https://bootstrap.pypa.io/bootstrap-buildout.py works here.

git clone https://github.com/collective/minimalplone4.git
cd minimalplone4
git checkout setuptools_39_error
wget https://bootstrap.pypa.io/bootstrap-buildout.py
python bootstrap-buildout.py
bin/buildout

[ida]

@idgserpro thanks for providing your recipe, executing it results in getting this message:

Upgraded:
  setuptools version 38.7.0;
restarting.

Which is repeated many times and then aborts with:

Error: Error downloading extends for URL https://dist.plone.org/release/4-latest/versions.cfg: <urlopen error [Errno -2] Name or service not known>

However I noticed that with the steps I described, the version-pin cannot take effect, when running buildout, because the error comes in the way before (checked for used vs. in bin/instance-script). Applying the patch and then running buildout will get the pinned version of setuptools, and then removing the patch again, no error occurs, so I can confirm the pin resolves the error.

[Rotonen]

Try with a different DNS server.

I've had dist.plone.org disappear and appear on the Google public DNS servers of 8.8.8.8 and 8.8.4.4, but remain stably visible with the Cloudflare public DNS server of 1.1.1.1. Seems to be some (hopefully) transient Cloudflare infrastructure issue, but I'm not 100% on that.

[ida]

@Rotonen, thanks for the hint, but by now I could confirm pinning works with the virtenv-procedure, if I apply the patch before buildouting and remove it afterwards. Now also found an explanation for that in this blog post of Tom Gross: https://blog.toms-projekte.de/workaround-setuptools-buildout-bug.html, as referenced in https://github.com/pypa/setuptools/issues/296, where I learned that "Buildout always fetches the latest version of setuptools for bootstraping. No matter what is defined in versions.cfg."

In any way the question still remains, if we want to fix the actual problem here instead of working around it with a pin?

My suggestion would be to add this line after the initial declaration of parsed:

if not isinstance(parsed, tuple):
    parsed = parsed.base_version.split('.')

[idgserpro]

Buildout always fetches the latest version of setuptools for bootstraping. No matter what is defined in versions.cfg.

Actually, nowadays it always download 33.1.1:

ez_setup.py is deprecated and when using it setuptools will be pinned to 33.1.1 since it's the last version that supports setuptools self upgrade/installation, check https://github.com/pypa/setuptools/issues/581 for more info; use pip to install setuptools

In any way the question still remains, if we want to fix the actual problem here instead of working around it with a pin?

The pin here is only as a reference (specially for minimalplone4), the idea here is really to "Support setuptools > 38.7.0".

[ida]

@idgserpro Great, does my proposition sound feasible to you or do you have a better suggestion?

[idgserpro]

I'm not that familiar with this package to say if it's the best approach, specially because we have to be careful about not breaking older setuptools releases...

[idgserpro]

@ida Not related to this issue, but is a good practice to reference to the snippets of code you plan to change, specially pointing to a specific commit (so the link will rarely break or lose it's contextual value in the future):

https://github.com/plone/plone.recipe.zope2instance/blob/d7354eb44dde06c73b968c9f211306257708137e/src/plone/recipe/zope2instance/__init__.py#L94

[pganssle]

It looks like setuptools is using an old, vendored version of packaging. If you're using a more recent version, you can do:

from packaging.version import parse as parse_version

parsed = parse_version.parse(dist.version)
major, minor, patch = parsed.version

[ida]

@idgserpro from future import reference_snippets :-) backwards-compat should be given, but the next quest that rises to me is how to release a 4-x-version here, if the newest is 5.x and breaks Plone-4-compat. I'm only used to incremental development upon the master-branch.

@pganssie Yes, setuptools includes packaging in its '_vendor'-directory, I'm not sure if pinning packaging in a version-config would work here, gonna leave it like it is and better live with the pin, thanks anyways!

[Rotonen]

Create two branches, one off master, one off the 4.x branch and do a PR for both.

https://github.com/plone/plone.recipe.zope2instance/tree/master https://github.com/plone/plone.recipe.zope2instance/tree/4.x

[ida]

@Rotonen that makes sense, Merci. So, the master doesn't have this problem, since it's hard-coding the Zope-vs to 4:

https://github.com/plone/plone.recipe.zope2instance/blob/master/src/plone/recipe/zope2instance/__init__.py#L93

My proposition for the 4.x-branch is this:

https://github.com/ida/plone.recipe.zope2instance/blob/fix37/src/plone/recipe/zope2instance/__init__.py#L96

But I wouldn't go for it without a seconder and otherwise pin setuptools in setup.py.

[mauritsvanrees]

The fix by @ida looks good and I am running Jenkins jobs now. Using newer methods frompkg_resources if available, as suggested by @pganssle, may work too, but for this limited use case (only figure out the version of Zope2), the simple fix seems the best one.

[idgserpro]

All green! Can we have a release? :)

[mauritsvanrees]

Already done just before you asked it. :-) I named it 4.4.0. Used in coredev 4.3, 5.0 and 5.1.

[djowett]

@mauritsvanrees Before they call - you answer! Reminds me of someone!