Actions: manage failing `cftime` dist. build w/ Ubuntu 18.04

[sadielbartholomew]

Working out a solution, and testing it via re-running the CI jobs, to bypass or fix the setup.py bdist_wheel build of cftime which fails for the Ubuntu 18.04 (only, i.e. not 20.04 too) distribution job. It seems to occur because distutils ends up being run, and that is old enough that it doesn't recognise a(t least one) metadata key in the cftime v.1.5.0 setup.py (namely "long_description_content_type").

Details

Specifically, the Actions log produced the following output:

Run pip3 install -e .
Obtaining file:///home/runner/work/cfunits/cfunits
Collecting cftime>=1.5.0 (from cfunits==3.3.4)
  Downloading https://files.pythonhosted.org/packages/c4/1f/a91a359605b4e67b57cc7170370ba24e86d800e8bd8260669165c8d7acce/cftime-1.5.1.1.tar.gz (46kB)
Collecting numpy>=1.15 (from cfunits==3.3.4)
  Downloading https://files.pythonhosted.org/packages/45/b2/6c7545bb7a38754d63048c7696804a0d947328125d81bf12beaa692c3ae3/numpy-1.19.5-cp36-cp36m-manylinux1_x86_64.whl (13.4MB)
Building wheels for collected packages: cftime
  Running setup.py bdist_wheel for cftime: started
  Running setup.py bdist_wheel for cftime: finished with status 'error'
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ppt2tcet/cftime/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp60vdbn0bpip-wheel- --python-tag cp36:
  /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
    warnings.warn(msg)
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.6
  creating build/lib.linux-x86_64-3.6/cftime
  copying src/cftime/__init__.py -> build/lib.linux-x86_64-3.6/cftime
  running build_ext
  building 'cftime._cftime' extension
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/tmp
  creating build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet
  creating build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet/cftime
  Failed building wheel for cftime
  creating build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet/cftime/src
  creating build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet/cftime/src/cftime
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/usr/lib/python3/dist-packages/numpy/core/include -I/usr/include/python3.6m -c /tmp/pip-build-ppt2tcet/cftime/src/cftime/_cftime.c -o build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet/cftime/src/cftime/_cftime.o
  x86_64-linux-gnu-gcc: error: /tmp/pip-build-ppt2tcet/cftime/src/cftime/_cftime.c: No such file or directory
  x86_64-linux-gnu-gcc: fatal error: no input files
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Running setup.py clean for cftime
Failed to build cftime
Installing collected packages: numpy, cftime, cfunits
  Running setup.py install for cftime: started
    Running setup.py install for cftime: finished with status 'error'
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ppt2tcet/cftime/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-tn2a91ay-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
      warnings.warn(msg)
    running install
    running build
    running build_py
    running build_ext
    building 'cftime._cftime' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/tmp
    creating build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet
    creating build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet/cftime
    creating build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet/cftime/src
    creating build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet/cftime/src/cftime
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/home/runner/.local/lib/python3.6/site-packages/numpy/core/include -I/usr/include/python3.6m -c /tmp/pip-build-ppt2tcet/cftime/src/cftime/_cftime.c -o build/temp.linux-x86_64-3.6/tmp/pip-build-ppt2tcet/cftime/src/cftime/_cftime.o
    x86_64-linux-gnu-gcc: error: /tmp/pip-build-ppt2tcet/cftime/src/cftime/_cftime.c: No such file or directory
    x86_64-linux-gnu-gcc: fatal error: no input files
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ppt2tcet/cftime/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-tn2a91ay-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-ppt2tcet/cftime/
Error: Process completed with exit code 1.

[sadielbartholomew]

Opening and closing to re-trigger the CI jobs...

[codecov-commenter]

Codecov Report

Merging #39 (ccf5d0d) into master (fb7982a) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #39   +/-   ##
=======================================
  Coverage   81.10%   81.10%           
=======================================
  Files           2        2           
  Lines         677      677           
=======================================
  Hits          549      549           
  Misses        128      128           

Flag	Coverage Δ
unittests	81.10% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update fb7982a...ccf5d0d. Read the comment docs.

[sadielbartholomew]

And again...

[sadielbartholomew]

OK the issue remains whatever cftime we use, so setuptools needs to be updated for the Ubuntu 18.04 job, though when I try that manually I get a different error:

  Running setup.py install for cftime: started
    Running setup.py install for cftime: finished with status 'error'
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-995k8piz/cftime/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-gvnwd773-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    running install
    /home/runner/.local/lib/python3.6/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      setuptools.SetuptoolsDeprecationWarning,

So not entirely sure what to do here. Let me try grabbing a later setuptools from pip, which is not ideal but I can't see what else can be done...

[sadielbartholomew]

Open-close to re-trigger again...

[sadielbartholomew]

OK updating setuptools via pip also doesn't work, and since I don't have time to debug this, for the sake of progress I will comment out the Ubuntu 18.04 dist-only run for now and create an issue to reinstate it when we know how.

We already test that cfunits can run via conda installations on that dist anyway, so we already have a good coverage of installation testing on the CI.

[sadielbartholomew]

I won't squash all of these commits together like I ordinarily would because they each show a solution I tried and that failed to get the cftime distribution to build given the older default setuptools, which will be useful to reference when we do get onto resolving the underlying issue.

Ultimately this PR now comments out the relevant lines to stop the Ubuntu 18.04 dist. job (only) from the un-test-suite_ubuntu_default_packages.yml workflow. I will merge it now and open the issue.