search

plone / buildout.coredev

Plone Core Development Buildout
http://docs.plone.org/develop/coredev/docs/
74 stars 75 forks source link

Pillow >= 8.0.0 required for running coredev on M1 Macs #708

Closed fredvd closed 1 year ago

fredvd commented 3 years ago

Plone 5.2 should support both Python2.7 and Python3.X , but Pillow < 8.0.0 cannot find zlib on Macs with the latest OS X and running on the new M1 architecture: 

  File "/var/folders/bc/qzs1h_3n2dggkdkm5t07_2m80000gn/T/easy_install-dm3pwbq3/Pillow-6.2.2/setup.py", line 687, in build_extensions
__main__.RequiredDependencyException: zlib

This was fixed in Pillow 8.0.0: (Added homebrew zlib include directory #4842 [radarhere]) But Pillow > 6.2.2 no longer supports Python 2.7

Could we maybe add a specific version override for python3: that pins Pillow to 8.00/8.1.0 for python3 but keeps it to 6.2.2 for python2?

tisto commented 3 years ago

Isn't 3.7 the recommended Python version for Plone 5.2? If this is the case maybe Python 2.7 should be the special case. Just thinking out loud here. :)

mauritsvanrees commented 3 years ago

A few months ago I tried to update Pillow from 6 to 8 in Plone coredev 6.0. This gave a few test failures. Probably fixable, but potentially tricky to make it work with both Pillow versions in 5.2. We should probably first fix in in Plone 6 and then see how hard it is to back port the fixes.

jensens commented 3 years ago

Meanwhile Pillow is kept at latest in 6.0 and it works fine AFAIK. If this is still an issue on 5.2: After M1 is newer than our 5.2 stack I would say the combination of Plone 5.2, Python 2.7 and M1 Mac is simply not supported.

mauritsvanrees commented 3 years ago

@jensens wrote:

After M1 is newer than our 5.2 stack I would say the combination of Plone 5.2, Python 2.7 and M1 Mac is simply not supported.

I think Fred already had enough trouble getting Python 2.7 itself installed properly on M1, let alone Pillow. I don't think anyone here wants to fix this particular combination. But it would be good if M1-Py3-Plone52 works. This apparently needs Pillow 8.

I should know better, but I tried again to upgrade Pillow on coredev 5.2, for Python 3 only. Basically:

[versions]
pillow = 8.3.1

[versions:python27]
pillow = 6.2.2

See branch maurits/pillow-8-on-52-py3

But some tests fail, among others PlonePAS. Jens already fixed this for Plone 6 in https://github.com/plone/Products.PlonePAS/pull/61. See my comment there on why I understand that this is for Plone 6 only. Shortly: the PR fixes it for Python 3, but breaks it for Python 2. Making it compatible with both is not fun to do.

And I think last time I tried it, there were failures in one or two other packages. Ah, plone.formwidget.namedfile fails with a different traceback than expected, that should be fixable. Probably https://github.com/plone/plone.formwidget.namedfile/pull/50 can be cherry-picked.

In other words: making Pillow 8 work on Plone 5.2 takes some effort, which is maybe not worth it.

@fredvd I wonder if pip install Pillow=6.2.2 does work on M1, because that could be a workaround. But I expect that this fails too.

fredvd commented 3 years ago

I’ll check what the current status is and if pip install has better results. Iirc you need Pillow 8 because it add extra paths/detection for compiling and linking the dependencies. They do this all the time also for major linux distributions where bundles image libraries slowly evolve over time.

For our projects I just pin Pillow in my (non git/vcs) buildout.cfg to 8 and forget about it.

the pain with buildout.coredev is more that buildout.cfg is in git. Or do we have something else for that in coredev?

mauritsvanrees commented 3 years ago

I have local.cfg in several of my coredev checkouts. This is in the .gitigore file. There is usually not much in it, but occasionally I add for example a hotfix there before it is published.

Rotonen commented 3 years ago

I'm currently also getting this on the 5.2 branch macOS 11.6 on Python 3.8.12 on a mid 2014 15" rMBP (i7-4980HQ).

Based on how Python itself needed to change to adapt to XCode SDK changes in macOS 11, I'd not say this is an ARM only consideration.

I'm referring to the fix they're refusing to backport to 3.6 at this point in time, leaving all versions of 3.6 unbuildable on macOS 11. I'd assume Pillow has done the same fix for their C extensions, which use zlib.

https://github.com/python/cpython/pull/25014 https://bugs.python.org/issue41100

fredvd commented 1 year ago

Closing this as newer versions of Pillow work fine on aarch64 by now.