search

ecmwf / cfgrib

A Python interface to map GRIB files to the NetCDF Common Data Model following the CF Convention using ecCodes
Apache License 2.0
407 stars 77 forks source link

Split cf2cdm out of cfgrib and move it to its own repo #353

Closed alexamici closed 1 year ago

alexamici commented 1 year ago

Is your feature request related to a problem? Please describe.

We at B-Open would like to use cf2cdm as a general tool, not tied to GRIB files, but being distributed as a part of cfgrib means it brings a baggage of heavy GRIB dependencies.

Note that I initially included cf2cdm in cfgrib because there was a CDS specific use case for translating coordinates of GRIB files to the CDS Common Data Model, but I was too lazy to create a separate package.

Judging from the folder history it has not been touched since Nov 2020:

https://github.com/ecmwf/cfgrib/commits/master/cf2cdm

In fact there is no use of cf2cdm in cfgrib itself, these are just two separate modules that happen to live in the same repository.

Describe the solution you'd like

I propose to split the cf2cdm module to its own repo, and remove it from cfgrib. We at B-Open have specific use cases for it and are willing to maintain it long term, so I propose to split it here: https://github.com/bopen/cf2cdm

In order not to fail existing users we can coordinate the first release of the new package on PyPI and conda and then add it as a dependency to cfgrib.

I proposed PR #354 to remove cf2cdm from cfgrib, but this is intended only as an illustration of what migration path I propose.

Describe alternatives you've considered

As an alternative we at B-Open could fork it with a different module name and you don't need to do anything.

Note that contributing to it as a part lf cfgrib is not really an option due to the above-mentioned issue with the GRIB dependency stack.

Additional context

No response

Organisation

B-Open

iainrussell commented 1 year ago

Hi @alexamici, nice to hear from you!

I have never used this functionality, nor been asked about it, so I'm happy to split if off. I'll just check though - @EddyCMWF and @JamesVarndell, is cf2cdm currently being used in CDS or CADS? If so, we would just need to be careful when moving versions.

Cheers, Iain

alexamici commented 1 year ago

@iainrussell nite to hear from you as well!

Indeed, cf2cdm is used by the legacy CDS Toolbox and I think it is used somewhere in the CADS codebase as well. In both cases the transition will mean that whenever the deployments update the cfgrib dependency (may never happen for the CDS Toolbox) the developers will need to add the dependency to the new cf2cdm package.

I can publish a v0.1 to PyPI immediately after you give me the OK. To have a conda-forge version I will need a couple more weeks.

EddyCMWF commented 1 year ago

Hi @alexamici , This is essentially the purpose of https://github.com/ecmwf-projects/cgul, I just never got round to making cfgrib dependant on cgul. I would be quite reluctant to have 2 packages doing the same thing, also I think cf2cdm is quite a misleading name. I am happy to accomondate your needs in cgul though to make it more operational. Eddy

alexamici commented 1 year ago

@EddyCMWF oh, I didn't connect the dots!

Indeed cgul aims to solve the same problem and I think I can close both this issue and the associated PR.

We are going to evaluate cgul for our needs and I'll leave managing the transition to you.