Moving to Python 3 > 2019-01-01

rsvp commented 6 years ago

Description of specific issue

Our code, currently compatible with both python27 and python3 series, is characterized as "straddling."

We are keen to use Python 3 to its full potential, and we currently accept the cost of writing cross-compatible code to allow a smooth transition, but we don’t intend to maintain this compatibility indefinitely. Although the transition has not been as quick as we hoped, we do see it taking place, with more and more people using, teaching and recommending Python 3.

The developers of the Python language extended support of Python 2.7 from 2015 to January 1, 2020, recognising that many people were still using Python 2. We believe that the extra 5 years is sufficient to transition off of Python 2, and our projects plan to stop supporting Python 2 when upstream support ends in 2020, if not before. We will then be able to simplify our code and take advantage of the many new features in the current version of the Python language and standard library.

We have pledged to drop support for Python 2.7 no later than 2020. Please see https://python3statement.org for details on the transition in the Python ecosystem.

[x] Deprecation notice: changes will be very gradual
[x] Enhancement: top-down refactoring of source code
[x] Support installation by pip, e.g. pip install --pre fecon236
[ ] Rebuild Docker container using Miniconda3-latest-Linux-x86_64.sh as base

Expected behavior

The transition should be smooth, and we will take the opportunity to refactor and re-organize the code from top-down. Users using the Anaconda distribution based on python27 will have to install a new distribution based on python3. For the fecon235 Docker image, the transition will be announced and automatically be set up for you.

Installation by pip

We have intentionally avoided the pip because optimal installation of fecon235 dependencies has been difficult, esp. for binaries and newbies. Unfortunately, pip is incapable of true dependency resolution (whereas Anaconda conda installs are reliable). This may change towards the year 2020, where only python3 is involved, and pipenv matures. The major annoyances are summarized here: https://python3statement.org/practicalities -- and minor details, not mentioned, are even more tedious.

fecon235 version: v6 notebooks will begin import of fecon236
Python version: al least 3.4 through 2019, expect at least 3.6 thereafter.
Operating systems: Linux, Mac, Windows

rsvp commented 6 years ago

Source code spin-off to fecon236

Home for our Jupyter notebooks shall remain at fecon235 due to their bulky size. Migration to the python3 kernel should present no problems since fecon235 is compatible with both python27 and python3. End users will be able to simply import fecon236 as fe as a drop-in replacement.

Version 10 of fecon236 represents the refactoring of only the fecon235 v5.18.0312 library code (not the Jupyter notebooks). It shall maintain compatibility with both python27 and python34.

After 2019-01-01, our official support for python27 will discontinue (like numpy and pandas), however, straddling code may still continue to work.

Version 11 of fecon236 will signal when our Travis builds under python27 fail, and at that point we expect to require at least Python 3.6.

Note the change in repository OWNERSHIP from INDIVIDUAL rsvp to ORGANIZATION MathSci for fecon236 in order to accommodate team permissions.

rsvp commented 6 years ago

Numpy and Pandas: Python 3 only > 2019-01-01

Starting January 1, 2019, all releases will be Python 3 only.

https://github.com/pandas-dev/pandas/pull/20540/files

Note that this date is one year before Python 3 itself. Since we depend heavily on numpy and pandas, fecon235 and fecon236 shall also follow this schedule.

Porting to Python 3 guide: https://docs.python.org/3/howto/pyporting.html

rsvp commented 6 years ago

Rebuild Docker container

The gist is to use Miniconda3 as base for python3, e.g.

wget http://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
bash miniconda.sh -b -p $HOME/miniconda
export PATH="$HOME/miniconda/bin:$PATH"
conda config --set always_yes yes --set changeps1 no
conda config --add channels pandas
conda update -q conda
conda info -a
conda install numpy pytz python-dateutil
conda install pandas=0.22
#  ^Example: install specific version.
conda install matplotlib
conda list

Then add the few dependencies from the scientific numpy/pandas stack. This avoids the huge bloat to the Docker image size from the alternate, but full, Anaconda distribution (astropy, bokeh, etc.). Take a look at https://hub.docker.com/r/rsvp/fecon235 where eventually we will directly install fecon236 modules.

(Also, this is the way the environment for fecon236 continuous integration will be built. The .travis.yml file will appear shortly with more details.)

rsvp commented 6 years ago

CHANGELOG 2018-06-23 (tag: v6.18.0623)

Major version change for fecon235 from v5 to v6 signaling our integration with fecon236 which was spun-off from our source code.

Henceforth, fecon235 becomes a repository solely of Jupyter notebooks. The old Python source code at fecon235 will remain for archival purposes, while new code development shifts over to fecon236. Please see https://git.io/econ for details.

Revise docs/fecon235-00-README.ipynb to introduce fecon236. Function names have been retained, but under fecon236 the call routing is expected to be more explicit than casual, i.e. modules names are more significant. Please see https://git.io/fecon-intro

rsvp / fecon235