ssec / sift

A visualization application for satellite imagery
http://sift.ssec.wisc.edu/
GNU General Public License v3.0
46 stars 14 forks source link

Fix Python 3.10 incompatibilities #331

Closed rayg-ssec closed 2 years ago

rayg-ssec commented 2 years ago

A number of python 3.10 gotchas including library location changes, type markup, Qt call signature types, float vs integer division prevent SIFT from starting up when using python 3.10.

djhoese commented 2 years ago

Looks like the website build errors are not caused by your changes:

docstring of uwsift.queue.TaskQueue.didMakeProgress:13:Inline emphasis start-string without end-string.

The regular test failures do seem related at least a little bit. Looks like pyqt was updated on conda-forge 2 months ago. Some/most of these errors could be a change in PyQt API which seems dumb (like setAlignment not taking the Alignment enum).

I don't see any issues on the pyqt feedstock about these issues so my assumption is that they are expected.

djhoese commented 2 years ago

For the record, here's what changed when I updated my environment (previous environment completely passed):

  sphinx                                         3.0.3-py_0 --> 4.4.0-pyh6c4a22f_1
  sphinxcontrib-htm~                             1.0.3-py_0 --> 2.0.0-pyhd8ed1ab_0
  sphinxcontrib-ser~                             1.1.4-py_0 --> 1.1.5-pyhd8ed1ab_1

But it didn't update pyqt/qt. this is a Python 3.7 environment so it must not be getting updates any more from conda-forge.

djhoese commented 2 years ago

And Python 3.9 environment when updating a few deps:

  pyproj                               3.1.0-py39ha9a7ae0_3 --> 3.3.0-py39hab5ddba_1
  pyqt                                5.12.3-py39hf3d152e_7 --> 5.12.3-py39hf3d152e_8
  pyqt-impl                           5.12.3-py39h0fcd23e_7 --> 5.12.3-py39hde8b62d_8
  pyqt5-sip                          4.19.18-py39he80948d_7 --> 4.19.18-py39he80948d_8
  pyqtchart                             5.12-py39h0fcd23e_7 --> 5.12-py39h0fcd23e_8
  pyqtwebengine                       5.12.1-py39h0fcd23e_7 --> 5.12.1-py39h0fcd23e_8
  pysocks                              1.7.1-py39hf3d152e_3 --> 1.7.1-py39hf3d152e_4
  python                           3.9.6-h49503c6_1_cpython --> 3.9.10-h85951f9_1_cpython
  pytz                                  2021.1-pyhd8ed1ab_0 --> 2021.3-pyhd8ed1ab_0
  pywavelets                           1.1.1-py39hce5d2b2_3 --> 1.2.0-py39hce5d2b2_1
  qt                                      5.12.9-hda022c4_4 --> 5.12.9-ha98a1a1_5
  rasterio                             1.2.6-py39hbc4e497_2 --> 1.2.10-py39h0401cea_4
  requests                              2.26.0-pyhd8ed1ab_0 --> 2.27.1-pyhd8ed1ab_0
  setuptools                          49.6.0-py39hf3d152e_3 --> 60.6.0-py39hf3d152e_0
  shapely                              1.7.1-py39ha61afbd_5 --> 1.8.0-py39ha65c37e_5
  snowballstemmer                        2.1.0-pyhd8ed1ab_0 --> 2.2.0-pyhd8ed1ab_0
  sphinx                                 4.1.2-pyh6c4a22f_1 --> 4.4.0-pyh6c4a22f_1
  sphinxcontrib-ser~                     1.1.5-pyhd8ed1ab_0 --> 1.1.5-pyhd8ed1ab_1

Both documentation and tests pass fine on my machine. Let's try an empty commit and see how tests do. I'd like them to pass on the existing 3.9 environment in CI before updating to 3.10.

djhoese commented 2 years ago

So I wasn't able to reproduce the issue locally, but then I realized that the conda packages being installed on CI were for Python 3.10. The conda environment was never being told what version of python to use so it was using the latest (Python 3.10), even though we thought we were testing 3.7 and 3.9. I'm updated the CI so it should be using 3.9 now. If that passes I will then build a local 3.10 environment and try to reproduce the issues we were seeing here in CI. I will then also update CI to run Python 3.7 and 3.10.

djhoese commented 2 years ago

Ok, this looks better. The failing test is expected as the current version of numba does not support the newest version of numpy and fails to import when it checks for that.

djhoese commented 2 years ago

@rayg-ssec So I think I've fixed everything now. The unstable test environment (last parameter is true) will remain failing until a new version of numba is released. I'm not sure of a good way around that.

As for the PyQt5 issues, I've described the issue in this vispy issue here:

https://github.com/vispy/vispy/issues/2212#issuecomment-1026993206

Bottom line is this seems to be a PyQt5 bug that, based on changelog entries, has been fixed in at least PyQt6. The URL for the changelog for PyQt5 is broken so I can't be sure if later versions of PyQt5 will have this fixed. It is not an issue with all setAlignment functions, only the QGroupBox from what I can tell. In Python 3.10, low-level implicit integer conversion is no longer done. In PyQt5 the AlignmentFlag object can be converted to an integer just fine, but the Alignment object resulting from |'ing of the AlignmentFlag objects does not appear to be able to handle this. The simplest workaround is to modify pyuic5 generated python files with an int wrapping the flag usage in setAlignment only for QGroupBox widgets.