different result for variable names when using store.get_open_data_params_schema(data_id) or store.describe_data(data_id)

xcube-dev / xcube-cds

An xcube plugin to generate data cubes from the Climate Data Store (CDS) API

MIT License

3 stars 1 forks source link

different result for variable names when using store.get_open_data_params_schema(data_id) or store.describe_data(data_id) #15

Closed AliceBalfanz closed 1 year ago

AliceBalfanz commented 4 years ago

When I take a look at the variable names of a dataset via the xcube-cds plugin, I get different results when using store.get_open_data_params_schema(data_id) and store.describe_data(data_id)

To reproduce:

from xcube.core.store import new_data_store

store = new_data_store('cds')
data_id = 'reanalysis-era5-single-levels-monthly-means:monthly_averaged_reanalysis'

store.get_open_data_params_schema(data_id)

store.describe_data(data_id)

Is this intentional? When I compare both ways of accessing data variables with xcube-sh plugin, the result is the same for both manners.

pont-us commented 4 years ago

The CDS API doesn't have unified variable names between the request and the result -- in some cases, e.g. in the soil moisture data set, there isn't even a one-to-one mapping between the two sets of variables. the CDS store plugin generally tries to provide a thin interface to the CDS API (i.e. manipulates the requests and results as little as possible). Of course it would be possible to expand the scope of the plugin to rename variables and unify the two schemes, but that's quite a lot of implementation and maintenance effort. We could discuss whether this ought to be a design goal for the plugin, but for now I'll close this issue as "by design".

AliceBalfanz commented 3 years ago

Reopening, because the behavior also affects the xcube generator and is really user-unfriendly. operational (old) xcube generator version (xcube Generation • UI v1.0.8 • API v1.0.12)

new version in stage xcube generator (xcube Generation • UI v1.1.0-dev.3 • API v1.0.13.dev1)

Furthermore it differs to the other stores, where cube generation request can be build by using the data_var.name provided within the store.describe_data(data_id)

pont-us commented 3 years ago

The new generator version simply looks incorrect to me: it's taking the variable names from describe_data, not from get_open_data_params_schema. I expect that this will also cause the request to fail. describe_data describes the structure of the returned data, not the structure of the request.

pont-us commented 3 years ago

After discussion today: still no clear resolution on how we should proceed with this, so I'm leaving the bug open but not actively working on it. Obviously we'll have to come to a decision before the next gen UI release, since this bug almost certainly breaks generation of CDS data cubes in the UI.

pont-us commented 1 year ago

Closing as duplicate of #72 . We're keeping the newer issue open in order to exclude the now-irrelevant discussion of the generator.