Closed seisman closed 8 months ago
Probably need to see if there is GMT 6.5.1 (https://github.com/GenericMappingTools/gmt/issues/8265).
Edit: Likely no. See https://github.com/GenericMappingTools/gmt/pull/8266.
~55 Failing tests from #2962 that need to be fixed:
Changing to ghostscript 10.02.1 (#2694) would increase the test failures from 55 to 210.
We can open 2 (or more) PRs to fix the failing tests:
@earth_relief_01d_p
now returns a 3 band image when loaded with rioxarray
instead of a 1 band image? Will need to update fixture_xr_image
in test_grdimage_image.py separately - #2963Since there are ~200 failures, I think it makes more sense to update baseline images by modules.
Since there are ~200 failures, I think it makes more sense to update baseline images by modules.
Maybe group some of them, otherwise there will be 35 PRs. This is the summary count:
Test | Count | PR |
---|---|---|
../pygmt/tests/test_basemap.py | 10 | #2964 @seisman |
../pygmt/tests/test_coast.py | 4 | #2965 @seisman |
../pygmt/tests/test_colorbar.py | 6 | #2981 |
../pygmt/tests/test_config.py | 10 | #2975 @seisman |
../pygmt/tests/test_contour.py | 6 | #2976 |
../pygmt/tests/test_datasets_earth_age.py | 2 | #2978 |
../pygmt/tests/test_datasets_earth_free_air_anomaly.py | 2 | #2978 |
../pygmt/tests/test_datasets_earth_geoid.py | 2 | #2978 |
../pygmt/tests/test_datasets_earth_magnetic_anomaly.py | 6 | #2978 |
../pygmt/tests/test_datasets_earth_relief.py | 10 | #2978 |
../pygmt/tests/test_datasets_earth_vertical_gravity_gradient.py | 2 | #2978 |
../pygmt/tests/test_geopandas.py | 9 | #2976 |
../pygmt/tests/test_grd2cpt.py | 1 | #2977 |
../pygmt/tests/test_grdcontour.py | 4 | #2976 |
../pygmt/tests/test_grdimage.py | 6 | #2984 |
../pygmt/tests/test_grdimage_image.py | 1 | #3005 |
../pygmt/tests/test_grdview.py | 1 | #2976 |
../pygmt/tests/test_helpers.py | 1 | #2976 |
../pygmt/tests/test_histogram.py | 2 | #2973 @seisman |
../pygmt/tests/test_inset.py | 2 | #2973 @seisman |
../pygmt/tests/test_legend.py | 3 | #2973 @seisman |
../pygmt/tests/test_logo.py | 2 | #2973 @seisman |
../pygmt/tests/test_makecpt.py | 8 | #2977 |
../pygmt/tests/test_meca.py | 16 | #2968 @seisman |
../pygmt/tests/test_plot.py | 22 | #2972 @michaelgrund |
../pygmt/tests/test_plot3d.py | 13 | #2971 @michaelgrund |
../pygmt/tests/test_rose.py | 7 | #2969 @michaelgrund |
../pygmt/tests/test_subplot.py | 5 | #2977 |
../pygmt/tests/test_ternary.py | 3 | #2973 @seisman |
../pygmt/tests/test_text.py | 24 | #2974 @seisman |
../pygmt/tests/test_tilemap.py | 4 | #2963 @weij14 |
../pygmt/tests/test_timestamp.py | 8 | #2970 @seisman |
../pygmt/tests/test_velo.py | 1 | #2973 @seisman |
../pygmt/tests/test_wiggle.py | 2 | #2973 @seisman |
../pygmt/tests/test_x2sys_cross.py | 3 | #2986 @weiji14 |
Yes, better to have < 10 baseline image changes to make PR review easier.
Just in case it may be helpful. Here are the steps I'm using to update baseline images:
pytest pygmt/tests/test_makecpt.py
results
directory and visually check the baseline.png
, result.png
and difference.png
(if any)result.png
are correct, run the following command to batch copy the result.png
to the pygmt/tests/baseline
directory (be careful that some tests share the same baseline images):
ls results | awk -F'.' '{printf("mv results/%s/result.png pygmt/tests/baseline/%s.png\n", $0, $4)}' | bash
dvc status
and dvc add ...
to add the changed baseline imagesgit add ...
and git commit
to commit the changed .dvc
filesdvc push
and git push
After PR #2971, #2972 and #2977, we're almost done with updating the baseline images. Here are some of the remaining failures.
Here is an example of the old and new baseline images for the test_colorbar_box
test. As you can see, the colorbar annotations are too small in the new baseline image, which makes me think it's an upstream bug. However, it's just because the colorbar is set to 1 cm long but 0.5 cm high, which I bet no one will use in their papers. So I think we should take the opportunity to think about how we should test colorbar. Since the colorbar
wrapper is just a simple wrapper with three lines, maybe we should just keep the simplest test and remove the others?
old | new |
---|---|
The grdimage failures are most likely caused by the recent updates of the earth relief dataset. I think we should take the opportunity to use the static earth relief dataset so that future dataset updates won't cause PyGMT failures.
The dataset-related failures are caused by the recent updates of GMT remote datasets. To fix these failures, we just need to get the correct min/max values of the new datasets, like what I'm doing in PR #2978. But, why do we check the min/max values of the remote datasets if it's fully out of our control?
grdimage
The grdimage failures are most likely caused by the recent updates of the earth relief dataset. I think we should take the opportunity to use the static earth relief dataset so that future dataset updates won't cause PyGMT failures.
Unfortunately we can't, because @static_earth_relief.nc
is not a global grid.
The x2sys_cross failures need also to be fixed, right @seisman ?
The x2sys_cross failures need also to be fixed, right @seisman ?
Yes, but I've never used them, so I'm unsure if the new results are correct.
The x2sys_cross failures need also to be fixed, right @seisman ?
Yes, but I've never used them, so I'm unsure if the new results are correct.
Let me look into that. A quick look at the changes to x2sys_cross.c
in GMT 6.5.0 only showed the addition of a nearest neighbour interpolation method to x2sys_cross
in https://github.com/GenericMappingTools/gmt/pull/7984, but that shouldn't have affected the results from the default linear interpolation method.
Some doctests also fail but are skipped in the "Tests" workflow.
$ export PYGMT_USE_EXTERNAL_DISPLAY=false && pytest pygmt/src/**.py
========================================== short test summary info ==========================================
FAILED pygmt/src/grd2xyz.py::pygmt.src.grd2xyz.grd2xyz
FAILED pygmt/src/grdclip.py::pygmt.src.grdclip.grdclip
FAILED pygmt/src/grdhisteq.py::pygmt.src.grdhisteq.grdhisteq.compute_bins
FAILED pygmt/src/grdproject.py::pygmt.src.grdproject.grdproject
FAILED pygmt/src/grdvolume.py::pygmt.src.grdvolume.grdvolume
FAILED pygmt/src/grd2xyz.py::pygmt.src.grd2xyz.grd2xyz
FAILED pygmt/src/grdclip.py::pygmt.src.grdclip.grdclip
FAILED pygmt/src/grdhisteq.py::pygmt.src.grdhisteq.grdhisteq.compute_bins
FAILED pygmt/src/grdproject.py::pygmt.src.grdproject.grdproject
FAILED pygmt/src/grdvolume.py::pygmt.src.grdvolume.grdvolume
We decided to skip some inline doctests to avoid slowing our tests in PR #1790, but these tests may be outdated and failures may be uncaught. We should:
:tada: GMT 6.5.0 has been released! It is installable from the conda-forge channel using the following command:
mamba install -c conda-forge gmt=6.5.0
To-Do for bumping the GMT version in CI:
environment.yml
ci/requirements/docs.yml
.github/workflows/benchmarks.yml
.github/workflows/cache_data.yaml
.github/workflows/ci_doctests.yaml
.github/workflows/ci_docs.yml
.github/workflows/ci_tests.yaml
.github/workflows/ci_tests_legacy.yaml
extlinks
urls indoc/conf.py
.github/workflows/check-links.yml