search

xcube-dev / xcube-cds

An xcube plugin to generate data cubes from the Climate Data Store (CDS) API
MIT License
3 stars 1 forks source link

Don't try to install conda-forge xcube in AppVeyor build #50

Closed pont-us closed 3 years ago

pont-us commented 3 years ago

Co-ordinated releases between xcube and xcube-cds (and other plugins) are complicated by the versioning: xcube-cds 0.x typically depends on xcube 0.x, but specifying xcube >= 0.x in the environment.yml breaks the build until the xcube 0.x conda-forge package becomes available -- meaning that xcube 0.x has to be both released and conda-packaged before xcube-cds 0.x can be tested.

The xcube-cds AppVeyor config already goes part of the way to resolving this problem: it actually installs xcube from the head of the master branch. However, before doing this, it also installs its own environment -- which includes the latest (and possibly as yet non-existent) xcube version -- then removes the conda-installed xcube. It would make more sense to do the following:

  1. Clone the xcube repository, create an xcube conda environment (containing xcube's dependencies but not xcube itself), and install xcube directly from the repository.
  2. Create a modified xcube-cds environment.yml which doesn't contain xcube as a dependency.
  3. Update the xcube environment using this modified environment.yml.
  4. Install xcube-cds from the repository.

This ensures that all xcube-cds's transitive dependencies through xcube are satisfied but avoids trying to install xcube itself through conda.

pont-us commented 3 years ago

It would be good to implement Issue #43 before fixing this one, in order to decrease the AppVeyor build time -- CI config modifications usually require at least a few debugging cycles, so a quicker CI build will save time when fixing the present issue.

TonioF commented 3 years ago

Another idea would be to come back to the use of maintenance and master branches. Changes that are incompatible with the last released xcube version would go into the master branch, others would go into both branches. The master branch could then hold a modified environment.yml permanently, until xcube is eventually released.

This procedure could/should also be extended to the xcube plugins.

pont-us commented 3 years ago

I think we discussed using a maintenance/master release model at some point recently and concluded that it would be a good idea, but only once xcube development has slowed down and stabilized somewhat. If/when we do go ahead with that, it would certainly help with the version skew problems between xcube and plugins in CI builds. But for now, I think the slightly hacky solution described above should be enough.