New minimum version of `matplotlib`

[davidhassell]

Hi @sadielbartholomew

It has been seen that sometimes cfplot seg faults for some versions of matplotlib when esmpy is also installed.

E.g., for me, esmpy==8.6.0 and matplotlib==3.8.0

I have found, however, that matplotlib==3.8.3 works with various versions of esmpy. So, could we make that the new minimum version requirement?

Thanks, David

[sadielbartholomew]

Thanks @davidhassell for the report of this. As discussed externally yesterday, I was not aware of this particular segmentation faulting before and I think your suggestion here sounds very wise.

I'll put in the update today and close this Issue to let you know it has been done, ready for the next release (date still TBC).

[sadielbartholomew]

Hi David. I looked into this earlier and sadly there is a complication in that setting as a minimum matplotlib v3.8.3 or above means Python 3.9 or above is required. I can tell this from applying the following diff to enforce to that updated version:

diff --git a/setup.py b/setup.py
index 7aa2522..2daf5ae 100644
--- a/setup.py
+++ b/setup.py
@@ -26,10 +26,10 @@ setup(
     package_dir = {"cfplot":"cfplot"},
     package_data = {"cfplot": package_data},
     include_package_data = True,
-    install_requires = ["matplotlib >=3.1.0",
-                        "cf-python >= 3.9.0",
-                        "scipy >= 1.4.0",
-                        "cartopy >= 0.17.0"
+    install_requires = ["matplotlib>=3.8.3",
+                        "cf-python>=3.9.0",
+                        "scipy>=1.4.0",
+                        "cartopy>=0.17.0"
                         ],
     url = "http://ajheaps.github.io/cf-plot",
     license = "LICENSE.txt",

after which upon attempted reinstallation, I see:

$ pip install -e .                                                            ─╯
Obtaining file:///home/slb93/git-repos/cf-plot
  Preparing metadata (setup.py) ... done
INFO: pip is looking at multiple versions of cf-plot to determine which version is compatible with other requirements. This could take a while.
ERROR: Ignored the following versions that require a different python version: 3.8.0 Requires-Python >=3.9; 3.8.0rc1 Requires-Python >=3.9; 3.8.1 Requires-Python >=3.9; 3.8.2 Requires-Python >=3.9; 3.8.3 Requires-Python >=3.9
ERROR: Could not find a version that satisfies the requirement matplotlib>=3.8.3 (from cf-plot) (from versions: 0.86, 0.86.1, 0.86.2, 0.91.0, 0.91.1, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.3.1, 1.4.0, 1.4.1rc1, 1.4.1, 1.4.2, 1.4.3, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0b1, 2.0.0b2, 2.0.0b3, 2.0.0b4, 2.0.0rc1, 2.0.0rc2, 2.0.0, 2.0.1, 2.0.2, 2.1.0rc1, 2.1.0, 2.1.1, 2.1.2, 2.2.0rc1, 2.2.0, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 3.0.0rc2, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.1.0rc1, 3.1.0rc2, 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0rc1, 3.2.0rc3, 3.2.0, 3.2.1, 3.2.2, 3.3.0rc1, 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4, 3.4.0rc1, 3.4.0rc2, 3.4.0rc3, 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.5.0b1, 3.5.0rc1, 3.5.0, 3.5.1, 3.5.2, 3.5.3, 3.6.0rc1, 3.6.0rc2, 3.6.0, 3.6.1, 3.6.2, 3.6.3, 3.7.0rc1, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.7.4, 3.7.5)
ERROR: No matching distribution found for matplotlib>=3.8.3

I believe cf-python still has a minimum Python version of 3.8, and I don't want to jump ahead of it.

Given the above, how about, for now, we work out the versions of matplotlib which seg fault with esmpy and exclude those in particular, instead of setting a minimum version? That way we can still keep some versions of it which allow for Python 3.8 (and possibly below also for cf-plot, I am yet to test which Python versions work and it is not well documented)?

[davidhassell]

Hi Sadie, why don't we push all of the cf* stable to Python 3.9? It's been the minimum version for the last two version of numpy, afterall.

[sadielbartholomew]

Good idea David, that sounds simpler and as you imply we wouldn't be too far in time from doing that anyway.

As agreed today, we won't do this until after we do our new release set of cf-python and cfdm, due to be done tomorrow.

[sadielbartholomew]

Now we have completed the release set, I can update the minimum version. However, first I would like to address #19 so that I have a single change log I can update when I add new features, make bug fixes, or, as in this case, update dependencies. So I will first do that. By the end of this week at the latest I should have both that and this task completed and will report back here.

[sadielbartholomew]

Note further context has been discovered on this: see https://github.com/cedadev/ceda-jaspy-envs/issues/138. I am holding on making the update here until I have upgraded all the NCAS-CMS Data Tools to minimum Python version 3.9 in turn after setting up a single-document cf-plot changelog, anyhow.

[sadielbartholomew]

Hi @davidhassell, I have had a little look at this, this morning. Unfortunately I still do see seg faults with matplotlib==3.8.3, however interestingly (you may have noticed yourself), the order of importing cf and cfplot can enable or disable the seg faulting, somehow:

Seg faults

Python 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cf
>>> import cfplot as cfp
>>> f = cf.read("../source/data/tas_A1.nc")
>>> cfp.con(f[0].subspace(time=15))
zsh: segmentation fault (core dumped)  python

so that's a work-around for folk, at least (I imagine it will be much more common for people to do the order of imports above, with cf first). I'll have a little explore in the matplotlib and esmpy Issue Trackers in case anyone has reported this environment behaviour and see if it is in the process of being fixed.

Works!

Python 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cfplot as cfp
>>> import cf
>>> f = cf.read("../source/data/tas_A1.nc")
>>> cfp.con(f[0].subspace(time=15))
/home/slb93/miniconda3/envs/cf-env-312/lib/python3.12/site-packages/cartopy/io/__init__.py:241: DownloadWarning: Downloading: https://naturalearth.s3.amazonaws.com/110m_physical/ne_110m_land.zip
  warnings.warn(f'Downloading: {url}', DownloadWarning)