Closed JimCircadian closed 2 years ago
Attempting to use this again under dev testing for #45
Some variables have come out, so I've re-enabled the node. These errors are transient and to be expected. Ultimately the mechanism for doing these lookups isn't ideal and too broadly selecting the endpoints, which is why I raised #43. Therefore this is worth closing and on a case-by-case basis, the -xs
switch needs to be used (e.g. -xs esgf.ceda.ac.uk -xs esgf-data2.diasjp.net
) to remove transiently down endpoints.
With the refactor for #43, we can make this more resilient, but only if the need warrants the development effort.
It needs to be noted that ESGF nodes are justifiable offline, rendering some variables unavailable (in this case MRI-ESM2-0 r3i1p1f1 ssp245 hus:
Query: {'source_id': 'MRI-ESM2-0', 'member_id': 'r3i1p1f1', 'frequency': 'day', 'variabl
e_id': 'hus', 'table_id': 'day', 'grid_label': 'gn', 'experiment_id': 'ssp245', 'data_node': 'esgf-data2.diasjp.net'}
[05-08-22 10:52:55 :DEBUG ] - Found ssp245: ['http://esgf-data2.diasjp.net/thredds/dodsC/esg_dataroot/CMIP6/ScenarioMI
P/MRI/MRI-ESM2-0/ssp245/r3i1p1f1/day/hus/gn/v20210830/hus_day_MRI-ESM2-0_ssp245_r3i1p1f1_gn_20150101-20241231.nc', 'http
://esgf-data2.diasjp.net/thredds/dodsC/esg_dataroot/CMIP6/ScenarioMIP/MRI/MRI-ESM2-0/ssp245/r3i1p1f1/day/hus/gn/v2021083
0/hus_day_MRI-ESM2-0_ssp245_r3i1p1f1_gn_20250101-20301231.nc', 'http://esgf-data2.diasjp.net/thredds/dodsC/esg_dataroot/
CMIP6/ScenarioMIP/MRI/MRI-ESM2-0/ssp245/r3i1p1f1/day/hus/gn/v20210830/hus_day_MRI-ESM2-0_ssp245_r3i1p1f1_gn_20310101-204
01231.nc', 'http://esgf-data2.diasjp.net/thredds/dodsC/esg_dataroot/CMIP6/ScenarioMIP/MRI/MRI-ESM2-0/ssp245/r3i1p1f1/day
/hus/gn/v20210830/hus_day_MRI-ESM2-0_ssp245_r3i1p1f1_gn_20410101-20501231.nc', 'http://esgf-data2.diasjp.net/thredds/dod
sC/esg_dataroot/CMIP6/ScenarioMIP/MRI/MRI-ESM2-0/ssp245/r3i1p1f1/day/hus/gn/v20210830/hus_day_MRI-ESM2-0_ssp245_r3i1p1f1
_gn_20510101-20601231.nc', 'http://esgf-data2.diasjp.net/thredds/dodsC/esg_dataroot/CMIP6/ScenarioMIP/MRI/MRI-ESM2-0/ssp
245/r3i1p1f1/day/hus/gn/v20210830/hus_day_MRI-ESM2-0_ssp245_r3i1p1f1_gn_20610101-20701231.nc', 'http://esgf-data2.diasjp
.net/thredds/dodsC/esg_dataroot/CMIP6/ScenarioMIP/MRI/MRI-ESM2-0/ssp245/r3i1p1f1/day/hus/gn/v20210830/hus_day_MRI-ESM2-0
_ssp245_r3i1p1f1_gn_20710101-20801231.nc', 'http://esgf-data2.diasjp.net/thredds/dodsC/esg_dataroot/CMIP6/ScenarioMIP/MR
I/MRI-ESM2-0/ssp245/r3i1p1f1/day/hus/gn/v20210830/hus_day_MRI-ESM2-0_ssp245_r3i1p1f1_gn_20810101-20901231.nc', 'http://e
sgf-data2.diasjp.net/thredds/dodsC/esg_dataroot/CMIP6/ScenarioMIP/MRI/MRI-ESM2-0/ssp245/r3i1p1f1/day/hus/gn/v20210830/hu
s_day_MRI-ESM2-0_ssp245_r3i1p1f1_gn_20910101-21001231.nc']
[05-08-22 10:52:55 :INFO ] - Found 16 hus results from ESGF search
...
RuntimeError: NetCDF: DAP server error
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <!DOCTYPE^ HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"><html><head><title>503 Service Temporarily Unavailable</title></head><body><h1>Service Temporarily Unavailable</h1><p>The server is temporarily unable to service yourrequest due to maintenance downtime or capacityproblems. Please try again later.</p></body></html>
Getting persistent DAP server errors from esgf.ceda.ac.uk, are we misusing this endpoint?
Discovery is via core nodes so a bit confusing: