search

Pyomo / pyomo

An object-oriented algebraic modeling language in Python for structured optimization problems.
https://www.pyomo.org
Other
2.03k stars 519 forks source link

Dropping Support for Python 2.7 #503

Closed whart222 closed 3 years ago

whart222 commented 6 years ago

This may seem premature, but many major packages in the Python community are planning to drop support for Python 2.7 in or before 2020. In fact, many major projects are planning to only support bug fixes sometime beforehand:

Since we often plan releases in early/mid-fall, I think it's reasonable to plan our last major release supporting Python 2.7 in the fall of 2019.

DLWoodruff commented 6 years ago

Very reasonable.

On Fri, May 18, 2018 at 12:23 AM, William Hart notifications@github.com wrote:

This may seem premature, but many major packages in the Python community are planning to drop support for Python 2.7 in or before 2020. In fact, many major projects are planning to only support bug fixes sometime beforehand:

Since we often plan releases in early/mid-fall, I think it's reasonable to plan our last major release supporting Python 2.7 in the fall of 2019.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Pyomo/pyomo/issues/503, or mute the thread https://github.com/notifications/unsubscribe-auth/APlZhGTNgAvBGVb-IkmkQOPhUyc5ugbgks5tznbWgaJpZM4UEP-b .

jsiirola commented 6 years ago

I think it is not reasonable to be planning dropping support for 2.7. 2.7 is still the default Python on most (if not all) Linux distributions, and as was pointed out in the 2.6 discussion, accounts for nearly 50% of our pip installs.

michaelbynum commented 6 years ago

Numpy’s plan for dropping support for Python2: https://docs.scipy.org/doc/numpy/neps/dropping-python2.7-proposal.html

On May 18, 2018, at 6:54 AM, John Siirola notifications@github.com<mailto:notifications@github.com> wrote:

I think it is not reasonable to be planning dropping support for 2.7. 2.7 is still the default Python on most (if not all) Linux distributions, and as was pointed out in the 2.6 discussion, accounts for nearly 50% of our pip installs.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/Pyomo/pyomo/issues/503#issuecomment-390197711, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ATdOLhvOrlTtouPF-7E06r9JklcI-vsCks5tzsR6gaJpZM4UEP-b.

whart222 commented 6 years ago

@jsiirola It seems that you are interpreting this issue as suggesting that we should drop Python 2.7 support today. I agree that we should not do that.

But a plan is design for future actions. But it's clear that the community is making concrete plans for dropping support in 2020. The timeline does, of course, relate to how this affects users. But it also affects developers; our recent decision to drop support for 2.6 was driven by the fact that other packages were dropping it.

My suggestion for the fall 2019 timeline was to be consistent with other packages' plans.

jsiirola commented 6 years ago

@whart222: Yes - sorry. I was confounding your statement above with @DLWoodruff's more aggressive timeline on the developers' list.

That said, I am still not sure why we want to "lead the charge" into removing support for 2.7. The projects that are leading that charge appear to be the ones that have compiled extensions (that is, extensions that are compiled against the Python API). This makes sense, as my understanding is supporting 2.x and 3.x is significantly harder when dealing with compiled code. For a Pure Python package like Pyomo(*), the burden is actually quite low -- so I am not sure why we want / need to be that aggressive.

Further, many of our users and platforms run significantly behind the development bleeding edge, so I can see us needing to support 2.7 past the community EOL.

(*) - yes, I realize that we are beginning to have a larger number of connections to compiled code (GJH, GSL, PyNumero). However, those interfaces are (intentionally) all though ctypes, so the external libraries are compiled completely independently of Python and have no dependence on Python version.

whart222 commented 6 years ago

@jsiirola I don't we'll be leading the charge, even if we follow numpy's timeline. But there are elements of Python 3.x that aren't backwards compatible. Support for the @ matrix operator is one that I'm particularly interested in exploiting soon.

whart222 commented 6 years ago

ALL: I've created a wiki page to start documenting planned changes when Python 2.7 support is removed. This will help us establish a timeline for making this transition.

https://github.com/Pyomo/pyomo/wiki/Planned-Changes-After-Dropping-Python-2.7

michaelbynum commented 6 years ago

I propose replacing dict with OrderedDict everywhere!

Michael

On May 21, 2018, at 7:58 PM, William Hart notifications@github.com<mailto:notifications@github.com> wrote:

ALL: I've created a wiki page to start documenting planned changes when Python 2.7 support is removed. This will help us establish a timeline for making this transition.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/Pyomo/pyomo/issues/503#issuecomment-390838443, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ATdOLkGS8JZHtfQByN6phja9dXBV7mbHks5t03C3gaJpZM4UEP-b.

michaelbynum commented 6 years ago

I also think we should revisit the possibility of making Var, Param, etc. hashable by setting __hash__ to object.__hash__. I know Gabe doesn't think it is safe (and he is probably correct), but I would like to be thoroughly convinced before we throw the idea out.

whart222 commented 6 years ago

@michaelbynum Isn't OrderedDict is slower? I think that's the reason we haven't already done that.

michaelbynum commented 6 years ago

@whart222 OrderedDict has a few operations that are slow in Python2. I think it is very efficient in Python3.

ghackebeil commented 6 years ago

OrderedDict is also the default storage used by *_dict containers in Kernel. I’ve never noticed any measurable performance issues relative to everything else that is done in Pyomo. On the other hand, I think the behavior you might want with OrderedDict comes for free with dict starting in Python 3.6 (and I think it’s part of the standard in 3.7).

hugovk commented 6 years ago

and as was pointed out in the 2.6 discussion, accounts for nearly 50% of our pip installs.

Python 2.7 accounted for 54% in January 2018: https://github.com/Pyomo/pyomo/issues/295#issuecomment-355252759.

Since then, it has dropped to 36%.

Here's the pip installs for pyomo from PyPI for May 2018:

python_version percent download_count
3.6 38.89% 714
2.7 35.84% 658
3.5 22.11% 406
3.4 2.83% 52
3.7 0.33% 6
Total 1,836

Source: pypinfo --start-date 2018-05-01 --end-date 2018-05-31 --percent --markdown pyomo pyversion

thorade commented 6 years ago

I think the idea e.g. in matplotlib or ipython is to have bug fix releases for python 2.7 until at least 2020, but add new features in the python 3 branch only. https://github.com/matplotlib/matplotlib/issues/9404 https://twitter.com/matplotlib/status/1042214628430958597 https://blog.jupyter.org/release-of-ipython-5-0-8ce60b8d2e8e https://blog.jupyter.org/release-of-ipython-6-0-2068cfb2c2a

hugovk commented 6 years ago

Both Matplotlib and IPython have pledged to drop Python 2 support in or before 2020:

http://python3statement.org/#sections30-projects

IPython before 2020, and Matplotlib until 2020-01-01:

http://python3statement.org/#sections40-timeline

whart222 commented 5 years ago

@carldlaird I think we should revisit this issue within the Pyomo Management Committee. We should have clear guidance for users about support for Python 2.7.

hugovk commented 5 years ago

You can see the drop off in PyPI downloads for 2.7 here:

https://pypistats.org/packages/pyomo

Also, NumPy and a number of Scientific Python projects have agreed on NEP 29, "Recommend Python and Numpy version support as a community policy standard":

https://numpy.org/neps/nep-0029-deprecation_policy.html

blnicho commented 4 years ago

At the PMC meeting yesterday we decided that we will be dropping support for Python 2.7 sometime in 2020.

michaelbynum commented 4 years ago

I think it is time to drop support for Python 2.7. Our support for Python 2.7 is preventing use of many features of Python 3 (e.g., type hinting, subprocess features, etc.). We should also discuss our plan for supporting Python 3.5. Python 3.6 was released 4 years ago, making Python 3.5 quite old.

hugovk commented 4 years ago

3.5 went EOL at the end of last month:

https://devguide.python.org/devcycle/#end-of-life-branches

michaelbynum commented 4 years ago

Thanks, @hugovk. We should probably put together a table like this in our documentation describing which version of Pyomo are compatible with which versions of Python. Something like

Pyomo Version Python Support
x.y.z >= 2.7
a.b.c >= 3.5

And so on.

whart222 commented 4 years ago

Since we already agreed to drop 2.7, if this is blocking new efforts then I agree the time is now.

michaelbynum commented 4 years ago

Is all we need to get this started a PR which removes tests for Python 2.7?

jsiirola commented 4 years ago

We had originally planned to drop 2.7 after 3.x became the default python on specific machines. Our most recent conversation indicated that there was no concrete timeline for that change. Given other workarounds, I don't think that we should continue to use that as a prerequisite for dropping 2.7.

One other thing came up today on a partner call: dropping python 2.7/3.4/3.5 support will have implications on the Gurobi direct interface. Specific versions of Gurobi only support specific versions of Python for the direct API:

Guroby version supported Python
5.6.3 2.7, 3.2
6.0.4 2.7, 3.2, 3.4
6.0.5 2.7, 3.2, 3.4
6.5.0 2.7, 3.4
6.5.2 2.7, 3.4, 3.5
7.0.0 2.7, 3.5
7.0.1 2.7, 3.5
7.0.2 2.7, 3.5
7.5.1 2.7, 3.6
7.5.2 2.7, 3.6
8.0.0 2.7, 3.5, 3.6
8.0.1 2.7, 3.5, 3.6
8.1.0 2.7, 3.5, 3.6, 3.7
8.1.1 2.7, 3.5, 3.6, 3.7
9.0.0 2.7, 3.5, 3.6, 3.7, 3.8
9.0.1 2.7, 3.5, 3.6, 3.7, 3.8

This came up because this particular partner was using a perpetual license for 6.5.0, so dropping 2.7 and 3.4 support would preclude their use of the direct interface.

michaelbynum commented 4 years ago

I would like to drop support for earlier versions of Gurobi in the direct interface anyway. Gurobi's lazy model updates are difficult to work with in versions before Gurobi 7. In many cases, this limits our ability to perform efficient model updates.

Older versions of Pyomo will always work, and the LP interface will always work.

jsiirola commented 4 years ago

@michaelbynum: seems reasonable to me.

The more I think about this, the more I think that this should be subsumed into a Pyomo 6.0 discussion where we: