IAMconsortium / pyam

Analysis & visualization of energy & climate scenarios
https://pyam-iamc.readthedocs.io/
Apache License 2.0
227 stars 118 forks source link

Support Python 3.12 #825

Closed glatterf42 closed 7 months ago

glatterf42 commented 7 months ago

Please confirm that this PR has done the following:

Description of PR

Supersedes #820 and adds support for Python 3.12.

Please note: this PR is based locally on top of #821, so we will need to rebase this onto main once that PR is merged.

danielhuppmann commented 7 months ago

@glatterf42 please rebase

glatterf42 commented 7 months ago

This PR is rebased on the latest main as indicated by the "9 files changes" and ~20 lines affected. Let me check the test failures.

glatterf42 commented 7 months ago

Even though I've run the exact same install command locally as in the GHA, namely pip install .[tests,optional_plotting,optional_io_formats,tutorials], I've noticed that there's a difference in the packages installed. Most prominently, pyam's version is registered differently: locally, I have 2.1.1.dev19+g589fc56 while it is pyam-iamc-0.1.dev1+geca1769 on the GitHub runner. I'm not sure this is important, but the pip install command justifies the installation of several packages from this version, e.g. Collecting bleach!=5.0.0 (from nbconvert>=7.3->pyam-iamc==0.1.dev1+geca1769). Here's a list of packages that are missing locally, but are installed on GitHub:

I don't know which one of these might be affecting the tests, but since my local install command didn't install them and all tests are passing, I'm guessing they are installed unintentionally. @danielhuppmann, @phackstock, any idea why this is happening? Could this be related to the bump of the setuptools minimum version?

~If it's not about these, there's just one version difference I found, though this should not affect the plotting capabilities, I think: dateutil has version 2.8.2 locally, but 2.9.0 on GHA. Let me rule this out quickly.~ EDIT: updated locally, tests are still successful, it's not about dateutil.

glatterf42 commented 7 months ago

So the step "Install dependencies and package" includes these lines:

   Preparing metadata (pyproject.toml): started
   Preparing metadata (pyproject.toml): finished with status 'done'

Even though there is no pyproject.toml on this branch. These lines also appear in the corresponding steps in the latest tests on the main branch, where there is no pyproject.toml either. And there, pyam is supposedly installed as v0.1.dev..., too. So this seems to have been a problem even before this PR.

When setting up a new project with poetry, it will ask you which version your project is one. Per default, this is 0.1.0, which would be written to pyproject.toml if you don't specify another one. My guess is that somehow, the automatically generated pyproject.toml contains this version or doesn't register use_scm_version correctly and thus also installs dependencies from v0.1.2, though this also seems somewhat unlikely.

Either way, I'm currently preparing a poetry-PR based on this branch, which will include a proper pyproject.toml, so the tests might then be passing.

danielhuppmann commented 7 months ago

@phackstock in #818 updated the GitHub-Action versions for checkout and setup, maybe that implied some default-to-toml routine?

glatterf42 commented 7 months ago

I'm just noticing that the packages listed above seem installed on my system after all, it's just that pytest tests/test_admin.py didn't print them.

phackstock commented 7 months ago

@phackstock in https://github.com/IAMconsortium/pyam/pull/818 updated the GitHub-Action versions for checkout and setup, maybe that implied some default-to-toml routine?

That was just an update to the latest versions and didn't break anything in that PR. So I don't think that that would break anything now. Plus @glatterf42 made the same updates in this PR to the other workflow files and they run fine.

phackstock commented 7 months ago

I'm just noticing that the packages listed above seem installed on my system after all, it's just that pytest tests/test_admin.py didn't print them.

Regarding the package version discrepancies that you reported, did you do your local installation from a fresh virtual environment or did you already have a previous version of pyam and its dependencies installed. If it's the latter that could explain the differences since the GH action always starts from scratch.

danielhuppmann commented 7 months ago

The failing tests have nothing to do with the pyam-version issue, that is just that @glatterf42 removed the matplotlib version pin.

phackstock commented 7 months ago

Ah ok I see

codecov[bot] commented 7 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.8%. Comparing base (c798e2c) to head (1ca323c).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #825 +/- ## ===================================== Coverage 94.8% 94.8% ===================================== Files 64 64 Lines 6092 6092 ===================================== Hits 5778 5778 Misses 314 314 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.