search

rsignell-usgs / ooici-erddap

https://erddap-uncabled.oceanobservatories.org/uncabled/erddap/index.html
0 stars 0 forks source link

100+ failing datasets on production ERDDAP #17

Closed rsignell-usgs closed 6 years ago

rsignell-usgs commented 6 years ago

ERDDAP provides a status page that lists failing datasets, and here on the OOI Production ERDDAP server there are more than 100 when I checked today: https://erddap-uncabled.oceanobservatories.org/uncabled/erddap/status.html These indicate some issues with the datasets.xml configuration, but luckily these are usually quite easy to fix.

The ERDDAP logs should contain info, or you can run ~webapps/content/ERDDAP/WEB-INF/DasDds.sh and enter the dataset ID to further diagnose the problems.

rsignell-usgs commented 6 years ago

I ran status just now (https://erddap-uncabled.oceanobservatories.org/uncabled/erddap/status.html) and still got 100+ failing datasets:

Current time is 2018-02-01T11:26:33+00:00
Startup was at  2018-01-05T03:08:46+00:00
Last major LoadDatasets started 31m 11s ago and is still running.
nGridDatasets  = 0
nTableDatasets = 1497
nTotalDatasets = 1497
n Datasets Failed To Load (in the last major LoadDatasets) = 140
    CE01ISSM-MFD35-05-PCO2WB000-pco2w_abc_dcl_instrument-telemetered-deployment0007-tabledap,
    CE01ISSM-MFD35-05-PCO2WB000-pco2w_abc_dcl_instrument-telemetered-deployment0008-tabledap,
    CE01ISSM-MFD35-06-PHSEND000-phsen_abcdef_dcl_instrument-telemetered-deployment0008-tabledap,
    CE01ISSM-MFD37-03-CTDBPC000-ctdbp_cdef_dcl_instrument-telemetered-deployment0008-tabledap,
    CE01ISSM-RID16-03-CTDBPC000-ctdbp_cdef_dcl_instrument-telemetered-deployment0008-tabledap,
rsignell-usgs commented 6 years ago

@kerfoot has access to the production logs now, identified the issue with 100+ failing datasets (failing on ioos_category) and also can duplicate the workflow of the production server, using the same input netcdf files and the same datasets.xml catalog.

John supplied a deployment request with a new datasets.xml on Friday, Jan 26, fixing problem with the ioos_category attribute (for now) by removing that parameter. This was deployed today on production today (Thursday, Feb 2).

Now we have only 3 failing datasets:

https://erddap-uncabled.oceanobservatories.org/uncabled/erddap/status.html returns:

Current time is 2018-02-01T16:02:33+00:00
Startup was at  2018-02-01T15:03:46+00:00
Last major LoadDatasets started 2m 10s ago and finished after 10 seconds.
nGridDatasets  = 0
nTableDatasets = 1634
nTotalDatasets = 1634
n Datasets Failed To Load (in the last major LoadDatasets) = 3
    CE02SHSM-SBD11-01-MOPAK0000-mopak_o_dcl_accel-telemetered-deployment0005-tabledap,
    CE07SHSM-SBD11-01-MOPAK0000-mopak_o_dcl_accel-telemetered-deployment0005-tabledap,
    CE09OSSM-SBD11-01-MOPAK0000-mopak_o_dcl_accel-telemetered-deployment0005-tabledap, (end)