iiasa / ixmp

The ix modeling platform for integrated and cross-cutting scenario analysis
https://docs.messageix.org/ixmp
Apache License 2.0
38 stars 112 forks source link

Run CI checks on Python 3.6 #359

Closed khaeru closed 4 years ago

khaeru commented 4 years ago

@OFR-IIASA flagged that TeamCity builds began to fail (e.g. #870) after the merge of iiasa/message_ix#286.

The issue here was that the message_ix GitHub Actions configuration did not correctly install Python 3.6 for the job that is supposed to use it. Syntax not compatible with Python 3.8 appeared to pass checks, when they should have failed. TeamCity uses Python 3.7, so TC builds began to fail when using the new code.

The same problem affects ixmp; see e.g. this log, supposedly for a py36 job, which contains:

Successfully setup CPython (3.8.5)

This PR fixes this issue.

I attempted to add Python 3.9-dev, but this fails because some dependencies, e.g. pandas, numpy, and llvmlite, are not available as precompiled binaries from PyPI for this as-yet-unreleased version of Python. pip on the job runner tries to compile them from source, but this requires a lot of dependencies that we don't have the resources to decipher. For now, the config for this Python version is commented out; but it should be enabled as soon as Python 3.9 is released, expected 2020-10-05.

iiasa/message_ix#381 will be expanded to include similar changes.

How to review

PR checklist

codecov[bot] commented 4 years ago

Codecov Report

Merging #359 into master will increase coverage by 0.00%. The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #359   +/-   ##
=======================================
  Coverage   96.32%   96.32%           
=======================================
  Files          44       44           
  Lines        4843     4845    +2     
=======================================
+ Hits         4665     4667    +2     
  Misses        178      178           
Impacted Files Coverage Δ
ixmp/core.py 95.46% <0.00%> (+0.01%) :arrow_up:
ixmp/backend/io.py 98.54% <0.00%> (+0.01%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update ac7e75e...95c6e54. Read the comment docs.

khaeru commented 4 years ago

The Windows / py3.6 check is failing at the step "Install R package and dependencies" with output like:

** testing if installed package can be loaded from temporary location
##[error]Error: package or namespace load failed for 'rixmp':
ERROR: loading failed
* removing 'D:/a/_temp/Library/rixmp'
 .onLoad failed in loadNamespace() for 'rixmp', details:
  call: py_module_import(module, convert = convert)
  error: ModuleNotFoundError: No module named 'ixmp.backend'

Detailed traceback: 
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\site-packages\ixmp\__init__.py", line 6, in <module>
    from .backend import BACKENDS, ItemType

##[error]Error: loading failed
Execution halted
##[error]Error: Failed to install 'rixmp' from local:
  (converted from warning) installation of package 'C:/Users/RUNNER~1/AppData/Local/Temp/RtmpyCyTV6/file128875cf1769/rixmp_3.0.0.9000.tar.gz' had non-zero exit status
Execution halted
##[error]Process completed with exit code 1.

A similar error occurs in a different step of this message_ix build, also for windows / py3.6

khaeru commented 4 years ago

Now the macOS builds, both versions, are failing in the "Install R packages and dependencies" step with output like:

ps       (1.3.3 -> 1.3.4) [CRAN]
rlang    (0.4.6 -> 0.4.7) [CRAN]
pkgbuild (1.0.8 -> 1.1.0) [CRAN]
Installing 3 packages: ps, rlang, pkgbuild
Installing packages into ‘/Users/runner/work/_temp/Library’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/bin/macosx/contrib/4.0/ps_1.3.4.tgz'
Error in download.file(url, destfile, method, mode = "wb", ...) : 
  (converted from warning) cannot open URL 'https://cloud.r-project.org/bin/macosx/contrib/4.0/ps_1.3.4.tgz': HTTP status was '404 Not Found'
Error in download.packages(pkgs, destdir = tmpd, available = available,  : 
  (converted from warning) download of package ‘ps’ failed
Calls: <Anonymous> ... force -> i.p -> .install.macbinary -> download.packages
Execution halted
##[error]Process completed with exit code 1.
khaeru commented 4 years ago

The Windows / py3.6 check is failing at the step "Install R package and dependencies"

This was resolved by excluding the job from the matrix. We don't have the resources to diagnose this at the moment. The windows-latest py3.8 job ensures that rixmp can be installed and run on Windows.

the macOS builds, both versions, are failing in the "Install R packages and dependencies" step

This was resolved by adding remotes::install_deps(..., upgrade = "never", ...) to prevent R looking for macOS binary packages to upgrade dependencies that are already installed. Failure to find these packages on CRAN was causing the check failure.