ESA-VirES / VirES-Python-Client

viresclient is a Python package for easy access to Swarm & Aeolus products as xarray.Dataset
https://viresclient.readthedocs.io
MIT License
14 stars 1 forks source link

Data conversion incompatible with cdflib #95

Closed jbwillis closed 11 months ago

jbwillis commented 1 year ago

cdflib recently released their v1.0 version. This included many breaking changes (see https://github.com/MAVENSDC/cdflib/releases/tag/1.0.0), and has introduced an error in viresclient when converting data to an xarray or a dataframe.

Code to reproduce the error

Based on the getting started example from here: https://notebooks.vires.services/notebooks/02a__intro-swarm-viresclient

(note this is copied from a jupyter notebook):

!pip install viresclient matplotlib numpy pandas xarray isoduration

from viresclient import set_token, SwarmRequest
import datetime
import matplotlib.pyplot as plt
import numpy as n

set_token("https://vires.services/ows")

# Set up connection with server
request = SwarmRequest()
# Set collection to use
request.set_collection("SW_OPER_MAGA_LR_1B")
# Set mix of products to fetch:
#   measurements (variables from the given collection)
#   models (magnetic model predictions at spacecraft sampling points)
#   auxiliaries (variables available with any collection)
# Reference: https://viresclient.readthedocs.io/en/latest/available_parameters.html
# Also set additional configuration such as:
#   sampling_step
request.set_products(
    measurements=["F"],
    sampling_step="PT10S",
    auxiliaries=["MLT"]
)
# Fetch data from a given time interval
data = request.get_between(
    start_time=dt.datetime(2016,1,1),
    end_time=dt.datetime(2016,1,2)
)

dx = data.as_xarray()

This results in the following error:

TypeError                                 Traceback (most recent call last)
[<ipython-input-9-f748d828bfa8>](https://localhost:8080/#) in <cell line: 1>()
----> 1 dx = data.as_xarray()

[/usr/local/lib/python3.10/dist-packages/viresclient/_data_handling.py](https://localhost:8080/#) in as_xarray(self, reshape)
    757         ds_list = []
    758         for i, data in enumerate(self.contents):
--> 759             ds_part = data.as_xarray(reshape=reshape)
    760             if ds_part is None:
    761                 print(

[/usr/local/lib/python3.10/dist-packages/viresclient/_data_handling.py](https://localhost:8080/#) in as_xarray(self, group, reshape)
    530             raise NotImplementedError("csv to xarray is not supported")
    531         elif self.filetype == "cdf":
--> 532             with FileReader(self._file) as f:
    533                 ds = f.as_xarray_dataset(reshape=reshape)
    534         elif self.filetype == "nc":

[/usr/local/lib/python3.10/dist-packages/viresclient/_data_handling.py](https://localhost:8080/#) in __init__(self, file, filetype)
    101             )
    102             self.data_filters = self._ensure_list(globalatts.get("DATA_FILTERS", []))
--> 103             self.variables = self._cdf.cdf_info()["zVariables"]
    104             self._varatts = {var: self._cdf.varattsget(var) for var in self.variables}
    105             self._varinfo = {var: self._cdf.varinq(var) for var in self.variables}

TypeError: 'CDFInfo' object is not subscriptable

Running pip show cdflib indicates that version 1.0.4 is installed.

Workaround

To work around this, I can specify the version of cdflib that pip installs

!pip install cdflib==0.4.9

With cdflib==0.4.9 installed, no error occurs when converting to xarray.

smithara commented 1 year ago

Thank you for the report and documenting the workaround. There is a fix ready so should be released soon. You can also install the fixed development version directly with pip install git+https://github.com/ESA-VirES/VirES-Python-Client.git@master

smithara commented 11 months ago

Fixed in v0.11.2