search

GenericMappingTools / pygmt

A Python interface for the Generic Mapping Tools.
https://www.pygmt.org
BSD 3-Clause "New" or "Revised" License
758 stars 220 forks source link

Test pygmt with GMT master branch #345

Closed seisman closed 4 years ago

seisman commented 5 years ago

Currently, we test pygmt with GMT 6.0.0rc4. I think we also need to test pygmt with the GMT master branch, to make sure changes in the core GMT don't break pygmt.

weiji14 commented 4 years ago

Continuing on the discussion from #397, I'm just throwing out some options here. In order of least to most work, do we want to:

1) Just test against master on Linux (i.e. only edit the .travis.yml file), basically #261 all over again. 2) Test on all platforms (Linux, MacOS, Windows and edit both .travis.yml and .azure-pipelines.yml 3) Refactor our test suite completely, and maybe consolidate our tests into one platform in one go: a) Follow core GMT and put everything in the azure-pipelines.yml file. See https://github.com/GenericMappingTools/gmt/pull/988. b) Go the Github Actions build matrix route, to test multiple platforms/Python versions. See also https://github.com/marketplace/actions/setup-miniconda.

As a side note, do we want to be testing for compatibility against multiple GMT versions in the long run (e.g. 6.0, 6.1, 6.2, etc?).

seisman commented 4 years ago

Ideally, we should test multiple GMT versions for compatibility. However, it seems difficult to maintain.

Sometimes, different GMT versions may produce slightly different images (e.g. a bug in 6.0.0, but fixed in 6.1.0), but the pygmt testing mechanism can only have one baseline image. Thus some pygmt tests must be failing for at least one GMT version.