Update dependencies 2024-07

[StevenMaude]

This should fix #362, but needs reviewing post-merge to confirm that the security alerts are cleared.

Dependabot fails to generate updates for several dependencies, so let's update everything manually.

[StevenMaude]

This doesn't yet update the test dependencies because that's a little more gnarly:

• The unit tests running in Docker use the OpenSAFELY Docker latest image which is actually Python 3.8.
• Even in the OpenSAFELY Docker v2 image, Python is still only at 3.10 and not the 3.11 that this project uses.
• Updating the Python image version and/or the test dependencies seems to make test_measures.py hang or at least very slow. This could also be due to dependencies in the Python image.

Edit: this was an active decision made in #40. But maybe this doesn't apply following the move of interactive into job-server?

[StevenMaude]

So I think my questions here reduce down to:

• Since interactive is now in job-server, should we be testing with the same environment as job-server?
• Is it sufficient to test with the same Python version (3.12), or do we need the full set of requirements, and some kind of more reproducible environment?

[StevenMaude]

Oh, the analysis tests are specifically for actions running with the Python image. But that also begs the question: we have two versions of the Python image; which one should be used, or both?

That also relates to #12.

[StevenMaude]

Running locally with the v2 Python image using Python 3.10, this takes much longer to run (more than 10 times longer) and results in:

FAILED tests/test_measures.py::test_calculate_total_counts - AssertionError: assert True == 'total'
FAILED tests/test_measures.py::test_calculate_group_counts - AssertionError: assert True == '2022-01-01'

Edit: I think this is related to .all() behaviour changes in pandas to be less quirky; see pandas-dev/pandas#12863. As specified in the documentation, even for the v1.0.3 version of pandas in the latest image, .all() is supposed to return whether all elements of the Series are True, but actually it ends up returning the value. We can fix this by checking equality with .eq instead of relying on this quirky behaviour.

The slowness of the tests is then Hypothesis generating examples, I think. With the fix, the tests run quickly in Python 3.8 or Python 3.10.

[StevenMaude]

In conclusion:

• updating the dependencies everywhere (dev, prod, test) should be fine
• I'll address the pandas quirkiness in another PR which will allow the tests to pass with a newer version of Python (#370)
• ~we should think about the test strategy~ it's fine because the analyses are templated, and currently use the image that we use in tests