Remote (OPeNDAP) access to COPERNICUS data is poorly

[jpolton]

Functionality to retrieve COPERNICUS data from OPeNDAP servers is not working as advertised in the COAsT: COPERNICUS docs (though maybe now moved and/or hidden)

import coast

# Authenticate with Copernicus and select a database.
database = coast.Copernicus(USERNAME, PASSWORD, "nrt")

# Instantiate a product with its ID
forecast = database.get_product("global-analysis-forecast-phy-001-024")

# Create a COAsT object with the relevant config file
nemo_t = coast.Gridded(fn_data=forecast, config="./config/example_cmems_grid_t.json")

I am not sure if the problem is to do with the plumbing their side or our side, or possibly even with the product names changing. I tried a few things but to no avail.

[MCazaly]

Picking this one up.

[MCazaly]

The URL we generate is https://nrt.cmems-du.eu/thredds/dodsC/global-analysis-forecast-phy-001-024.dds, which (when authenticated) gives us a 200 OK but an otherwise empty response. Interestingly, if I add some nonsense to that URL (i.e. https://nrt.cmems-du.eu/thredds/dodsC/this-is-probably-not-a-real-dataset.dds) I get the exact same response with a 200 OK. This would likely indicate either an issue with the URL (the asset location/name may have changed) or an issue on Copernicus' end.

@jpolton Do you have a time-frame for when this issue first appeared?

[MCazaly]

Taking a look at https://data.marine.copernicus.eu/product/GLOBAL_ANALYSISFORECAST_PHY_001_024/services, global-analysis-forecast-phy-001-024 is nowhere to be seen.

If I take the first listed dataset (cmems_mod_glo_phy-cur_anfc_0.083deg_PT6H-i) and pass that into our client, I get the dataset back as expected.

@jpolton I suppose then the question is which dataset you want to offer as an example? Would it be worth asking Copernicus if they can provide permalinks to CMEMS datasets?

In the meantime, I'll see if we can't catch this error in future and produce a friendlier error message.

[MCazaly]

See https://github.com/British-Oceanographic-Data-Centre/COAsT-site/pull/98 for the fix.