search

collective / buildout.python

Buildout all the Pythons
109 stars 47 forks source link

Add `--enable-optimizations` configure flag. #57

Closed hannosch closed 7 years ago

hannosch commented 7 years ago

This enables profile guided optimizations (PGO) on supported Python versions. PGO increases build time significantly, but results in Python being faster at runtime.

The configure flag is backported to the 2.7, 3.5 and 3.6 branches, but only available in the last 3.6 release as of yet. Once 2.7.13 and 3.5.3 are released, it will take effect for those as well. See for example http://bugs.python.org/issue26359.

There are a whole bunch of other related issues around LTO (link time optimization), but so far LTO is considered to be too unstable by the Python developers to be enabled with the new flag.

One can always remove this setting via extra_options -= --enable-optimizations or overwriting all of extra_options. Right now adding the flag for 2.7/3.5 results in a harmless configure warning. But I'd rather add the flag now before I forget about it again ;)

@mjpieters, @fschulze Thoughts?

mjpieters commented 7 years ago

Faster Python interpreters after a slightly longer wait to produce one? Sounds like a great plan to me! buildout.python is meant to be used in production environments after all.

However, could the README be updated to state this explicitly? Builds are optimised to ensure best performance; if such optimisations are not needed, disable them with ..., or words to that effect.

hannosch commented 7 years ago

@mjpieters good idea! I've added one more commit, updating the readme.

fschulze commented 7 years ago

a belated +1

@mjpieters personally I see buildout.python for development environments, not production. In production it's pretty rare to need all kinds of Python versions. But I kinda use it in production on my own Laptop :)

mjpieters commented 7 years ago

But don't people include the recipes in their production environments to build a specific Python version?

mauritsvanrees commented 7 years ago

On production I use it for one or two clients where the official Python packages on the server are too old.

aclark4life commented 7 years ago

Hmmm, a belated comment: I'd say buildout.python is meant to "build all the Pythons" for wherever you find that useful. As it so happens, I only ever use on my laptop (to build Py2.4 to run old Plones) but I could see supporting production environments too.

fschulze commented 7 years ago

Unfortunately Python 2.7.13 fails to build for me with this flag on El Capitan. I can't find a proper reason. The _struct and several other C extension modules fail to build.

mjpieters commented 7 years ago

2.7.13 fails to build on macOS sierra too.