Update testing environments

OriolAbril commented 3 months ago

Description

The initial goal of the PR was to test if ArviZ was already compatible with numpy 2.0 (which had just been pre-released as a beta version). Several dependencies weren't compatible yet, so it wasn't feasible to do. While waiting for this, we also saw ArviZ wasn't compatible with the latest scipy pre-release https://github.com/arviz-devs/arviz/issues/2330, and the fix wasn't released before the actual release, generating many more issues like https://github.com/arviz-devs/arviz/issues/2336. Moreover, even though the goal is to follow SPEC 0 guidance on dependency minimum version, having numba as optional dependency in fact pushes us the other way around, it often takes months after the release of a minor python version until we can incorporate it into our CI.

To try and handle all this, this PR updates our testing environments from 4 to 5. Now the environments are:

3 envs: "latest" 3 pythons with all optional dependencies, we ensure all test suite is executed. "latest" won't always be true, between a python release and numba being compatible with it the latest python version won't be there.
1 env: latest python with no optional dependencies. now always latest. Not all test suite is executed, any test that requires a missing dependency is skipped.
1 env: 2nd to last python released with no optional dependencies and nightly versions for scientific python packages. This should give us a significantly larger window for changes in dependencies that break ArviZ and for some reason go unnoticed.

Checklist

[x] Follows official PR format
[x] Includes new or updated tests to cover the new feature
[x] Code style correct (follows pylint and black guidelines)
[ ] Changes are listed in changelog

📚 Documentation preview 📚: https://arviz--2321.org.readthedocs.build/en/2321/

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 86.98%. Comparing base (ab82f80) to head (8e8feaf).

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #2321 +/- ## ========================================== + Coverage 86.84% 86.98% +0.14% ========================================== Files 123 123 Lines 12745 12744 -1 ========================================== + Hits 11068 11085 +17 + Misses 1677 1659 -18 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

OriolAbril commented 3 months ago

Should be ready to merge!

OriolAbril commented 2 months ago

So the current setup tests some github repo too, but we would skip these?

The preview job currently installs xarray from github. With this change it won't install from github but it will install nigthly versions for xarray, numpy, scipy, h5py, matplotlib... So when it comes to xarray the behaviour is quite similar, at most testing on a day old nightly instead of exact present moment github version. But for all other scientific python libraries we will catch breaks and issues way earlier than we did before.

arviz-devs / arviz