ecmwf / earthkit-data

A format-agnostic Python interface for geospatial data
Apache License 2.0
57 stars 17 forks source link

NetCDFFieldList does not have a write method #194

Closed EddyCMWF closed 1 year ago

EddyCMWF commented 1 year ago

What happened?

It is not possible to save netCDFFieldList objects using the .save method, e.g.: ek_data_object.save('output.nc'). This is often necessary when working with remote files, e.g. from sources "url" or "cds".

Note this is not a problem with the NetCDFReader object.

What are the steps to reproduce the bug?

import earthkit as ek
from earthkit.data.testing import earthkit_remote_test_data_file
ek_data = ek.data.from_source(
    "url",
    earthkit_remote_test_data_file("examples/efas.nc")
)
ek_data.save('temp.nc')

Version

latest

Platform (OS and architecture)

MacOS + Linux

Relevant log output

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[7], line 5
      1 ek_data = ek.data.from_source(
      2     "url",
      3     earthkit_remote_test_data_file("examples/efas.nc")
      4 )
----> 5 ek_data.save('temp.nc')

File ~/Work/Git_Repositories/EARTHKIT/earthkit-data/earthkit/data/core/fieldlist.py:1169, in FieldList.save(self, filename, append)
   1167 flag = "wb" if not append else "ab"
   1168 with open(filename, flag) as f:
-> 1169     self.write(f)

File ~/Work/Git_Repositories/EARTHKIT/earthkit-data/earthkit/data/core/fieldlist.py:1180, in FieldList.write(self, f)
   1172 r"""Write all the fields to a file object.
   1173 
   1174 Parameters
   (...)
   1177     The target file object.
   1178 """
   1179 for s in self:
-> 1180     s.write(f)

File ~/Work/Git_Repositories/EARTHKIT/earthkit-data/earthkit/data/core/__init__.py:74, in Base.write(self, f)
     71 @abstractmethod
     72 def write(self, f):
     73     """Writes data to the ``f`` file object."""
---> 74     self._not_implemented()

File ~/Work/Git_Repositories/EARTHKIT/earthkit-data/earthkit/data/core/__init__.py:122, in Base._not_implemented(self)
    120 if hasattr(self, "path"):
    121     extra = f" on {self.path}"
--> 122 raise NotImplementedError(f"{module}.{name}.{func}(){extra}")

NotImplementedError: earthkit.data.readers.netcdf.NetCDFField.write()

Accompanying data

No response

Organisation

No response

sandorkertesz commented 1 year ago

I do not consider this as a bug, but something that is not yet implemented. It is not trivial to write a NetCDFieldList to disk since it can contain an arbitrary subset of the "fields".

EddyCMWF commented 1 year ago

Okay, I wasn't sure whether to tag as bug or enhancement, I am happy either way. My solution is to point to the existing to_netcdf method. Probably best to carry on this conversation on the PR