Update build to support Python 3.8

[brahmaneya]

Description of proposed changes

Build config updates:

• Add Python 3.8 build to Travis
• Remove unneeded coverage and documentation checks for >3.6

Package version updates:

• Clamp NumPy to a max of 1.20 which introduces typing and other incompatible changes
• Clamp blis, which requires newer Python versions for latest releases
• Pin Dask and distributed requirements to pull versions before version scheme change (now YYYY.MM)
• Bump scikit-learn version upper bound to allow recent versions (which correctly has a Cython dependency for isolated wheel builds on Python 3.8+)

Fixes #1627

Test plan

• Travis runs pass for 3.6, 3.7, 3.8
• Will manually monitor full trunk runs once this lands

Checklist

Need help on these? Just ask!

• [x] I have read the CONTRIBUTING document.
• [x] I have updated the documentation accordingly.
• [x] I have added tests to cover my changes.
• [x] I have run tox -e complex and/or tox -e spark if appropriate.
• [x] All new and existing tests passed.

[rth]

(which correctly has a Cython dependency)

For information, the issue not a Cython dependency, but that for python 3.8 users are trying to build scikit-learn 0.22 from sources since binary wheels do not exist for that Python version. And there yes, it looks like Cython should have been declared a build dependency but it wasn't.

[rth]

BTW, if you want to auto-close the associate issue the PR description needs to say "Fixes #1620" (there is a formatting issue currently).

[rjurney]

(which correctly has a Cython dependency)

For information, the issue not a Cython dependency, but that for python 3.8 users are trying to build scikit-learn 0.22 from sources since binary wheels do not exist for that Python version. And there yes, it looks like Cython should have been declared a build dependency but it wasn't.

Building Cython doesn't actually fix the problem, however.

[rth]

Building Cython doesn't actually fix the problem, however.

You shouldn't need Cython to install a recent version of scikit-learn . What issue are you experiencing? If I install this branch of snorkel on Python 3.8

conda create -n test-env python=3.8 scipy
conda activate test-env
pip install https://github.com/snorkel-team/snorkel/archive/sklearn_bump.zip

it works fine for me:

``` Downloading https://github.com/snorkel-team/snorkel/archive/sklearn_bump.zip \ 1.5 MB 3.0 MB/s Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Requirement already satisfied: numpy<2.0.0,>=1.16.0 in ./miniconda3/envs/scipy-dev/lib/python3.8/site-packages (from snorkel==0.9.6+dev) (1.20.1) Collecting munkres>=1.0.6 Downloading munkres-1.1.4-py2.py3-none-any.whl (7.0 kB) Collecting networkx<2.4,>=2.2 Downloading networkx-2.3.zip (1.7 MB) |████████████████████████████████| 1.7 MB 19.2 MB/s Collecting decorator>=4.3.0 Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB) Collecting pandas<2.0.0,>=0.25.0 Downloading pandas-1.2.2-cp38-cp38-manylinux1_x86_64.whl (9.7 MB) |████████████████████████████████| 9.7 MB 76.6 MB/s Collecting python-dateutil>=2.7.3 Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB) Collecting pytz>=2017.3 Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB) |████████████████████████████████| 510 kB 100.7 MB/s Collecting scikit-learn<0.25.0,>=0.20.2 Using cached scikit_learn-0.24.1-cp38-cp38-manylinux2010_x86_64.whl (24.9 MB) Collecting joblib>=0.11 Downloading joblib-1.0.1-py3-none-any.whl (303 kB) |████████████████████████████████| 303 kB 112.5 MB/s Collecting scipy<2.0.0,>=1.2.0 Using cached scipy-1.6.0-cp38-cp38-manylinux1_x86_64.whl (27.2 MB) Collecting six>=1.5 Using cached six-1.15.0-py2.py3-none-any.whl (10 kB) Collecting tensorboard<2.0.0,>=1.14.0 Downloading tensorboard-1.15.0-py3-none-any.whl (3.8 MB) |████████████████████████████████| 3.8 MB 92.4 MB/s Requirement already satisfied: wheel>=0.26 in ./miniconda3/envs/scipy-dev/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel==0.9.6+dev) (0.36.2) Requirement already satisfied: setuptools>=41.0.0 in ./miniconda3/envs/scipy-dev/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel==0.9.6+dev) (52.0.0.post20210125) Collecting absl-py>=0.4 Downloading absl_py-0.11.0-py3-none-any.whl (127 kB) |████████████████████████████████| 127 kB 107.3 MB/s Collecting grpcio>=1.6.3 Downloading grpcio-1.35.0-cp38-cp38-manylinux2014_x86_64.whl (4.1 MB) |████████████████████████████████| 4.1 MB 99.6 MB/s Collecting markdown>=2.6.8 Downloading Markdown-3.3.3-py3-none-any.whl (96 kB) |████████████████████████████████| 96 kB 6.4 MB/s Collecting protobuf>=3.6.0 Downloading protobuf-3.14.0-cp38-cp38-manylinux1_x86_64.whl (1.0 MB) |████████████████████████████████| 1.0 MB 98.6 MB/s Collecting threadpoolctl>=2.0.0 Using cached threadpoolctl-2.1.0-py3-none-any.whl (12 kB) Collecting torch<2.0.0,>=1.2.0 Downloading torch-1.7.1-cp38-cp38-manylinux1_x86_64.whl (776.8 MB) |████████████████████████████████| 776.8 MB 20 kB/s Collecting tqdm<5.0.0,>=4.33.0 Downloading tqdm-4.56.2-py2.py3-none-any.whl (72 kB) |████████████████████████████████| 72 kB 1.3 MB/s Collecting werkzeug>=0.11.15 Using cached Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB) Collecting typing-extensions Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB) Building wheels for collected packages: snorkel, networkx Building wheel for snorkel (PEP 517) ... done Created wheel for snorkel: filename=snorkel-0.9.6+dev-py3-none-any.whl size=145128 sha256=bccc603807dbbee4a4827dafff6fac5d14e85bdb29b73defe73debd45a042268 Stored in directory: /tmp/pip-ephem-wheel-cache-jjuqwci_/wheels/37/0e/82/c9e52f544e98eed20b9c6dd50df8842828bdafcf6819c658c6 Building wheel for networkx (setup.py) ... done Created wheel for networkx: filename=networkx-2.3-py2.py3-none-any.whl size=1555990 sha256=c69626ad1eab9ba996f64d9b85d0d7c045af9ba2f19f8f385ff6d5b37b52a497 Stored in directory: /home/rth/.cache/pip/wheels/ff/62/9e/0ed2d25fd4f5761e2d19568cda0c32716556dfa682e65ecf64 Successfully built snorkel networkx Installing collected packages: six, werkzeug, typing-extensions, threadpoolctl, scipy, pytz, python-dateutil, protobuf, markdown, joblib, grpcio, decorator, absl-py, tqdm, torch, tensorboard, scikit-learn, pandas, networkx, munkres, snorkel Successfully installed absl-py-0.11.0 decorator-4.4.2 grpcio-1.35.0 joblib-1.0.1 markdown-3.3.3 munkres-1.1.4 networkx-2.3 pandas-1.2.2 protobuf-3.14.0 python-dateutil-2.8.1 pytz-2021.1 scikit-learn-0.24.1 scipy-1.6.0 six-1.15.0 snorkel-0.9.6+dev tensorboard-1.15.0 threadpoolctl-2.1.0 torch-1.7.1 tqdm-4.56.2 typing-extensions-3.7.4.3 werkzeug-1.0.1 ```

[codecov[bot]]

Codecov Report

Merging #1628 (af6f152) into master (ed77718) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1628   +/-   ##
=======================================
  Coverage   97.21%   97.21%           
=======================================
  Files          68       68           
  Lines        2151     2151           
  Branches      345      345           
=======================================
  Hits         2091     2091           
  Misses         31       31           
  Partials       29       29           

[henryre]

As a note: we turned on weekly cron builds to detect issues from changing dependency on an ongoing basis

[rjurney]

@henryre thanks!