NOAA OFS Regular Grid netCDF files and ncWMS2

[patrick-tripp]

I’ve been unable to get ncWMS2 to read the OFS regular grid data. ncWMS2 does not recognize any variables. I have tested ncWMS2 2.4.1 up to 2.5.0. I have tried everything I can think of to get this data recognized to no avail. Any help with this is appreciated.

The current version (5.4.2) of NetCDF-Java and the Tools-UI application have no problem reading and displaying the data. https://www.unidata.ucar.edu/downloads/netcdf-java/

The older version of WMS being run at CO-OPS and the Godiva2 interface also work fine as seen below.

Here is the sample data used for this demonstration:

Regular Grid (CO-OPS only retains the most recent three days of regulargrid data): https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/CBOFS/MODELS/2021/09/27/nos.cbofs.regulargrid.f001.20210927.t00z.nc https://opendap.co-ops.nos.noaa.gov/thredds/godiva2/godiva2.html?server=https://opendap.co-ops.nos.noaa.gov/thredds/wms/NOAA/CBOFS/MODELS/2021/09/27/nos.cbofs.regulargrid.f001.20210927.t00z.nc

ROMS Grid: https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/CBOFS/MODELS/2021/09/28/nos.cbofs.fields.f001.20210928.t00z.nc https://opendap.co-ops.nos.noaa.gov/thredds/godiva2/godiva2.html?server=https://opendap.co-ops.nos.noaa.gov/thredds/wms/NOAA/CBOFS/MODELS/2021/09/27/nos.cbofs.fields.f001.20210927.t00z.nc

This is easily reproduced in the standalone ncWMS2 application that is available here: https://github.com/Reading-eScience-Centre/ncwms/releases

Note there are is no "+" to expand the list of variables for CBOFS RG CO-OPS.

Here are the config files I am using to demonstrate the issue. CO-OPS only retains the most recent three days of regulargrid data.

CBOFS-COOPS.xml

<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" location="https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/CBOFS/MODELS/2021/09/27/nos.cbofs.fields.f001.20210927.t00z.nc">
</netcdf>

CBOFSRG-COOPS.xml

<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" 
location="https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/CBOFS/MODELS/2021/09/27/nos.cbofs.regulargrid.f001.20210927.t00z.nc">
    <!-- changes to variables below -->
    <variable name="mask">
        <attribute name="flag_values" type="byte" value="0 1"/>
        <attribute name="flag_meanings" value="land water"/>
    </variable>
    <variable name="salt">
        <attribute name="units" type="String" value=""/>
    </variable>
</netcdf>

Thank you.

Patrick Tripp Senior Software Engineer RPS | North America

[guygriffiths]

The latest version of the regular grid is certainly showing data for me (this is just updating your URL with today's date): https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/CBOFS/MODELS/2021/10/05/nos.cbofs.fields.f001.20211005.t00z.nc

As is the ROMS data which you linked to: https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/CBOFS/MODELS/2021/09/28/nos.cbofs.fields.f001.20210928.t00z.nc

Putting those URLs in the location field of ncWMS2 (both on Tomcat or standalone - v2.5.0) generates what looks like mostly sensible data - although there are some errant points compared to your screenshot of the earlier version. Are you getting any console errors when you run into these issues?

(Note that there are issues with the default background map at the moment - that's what the pink missing images are in your screenshots.)

[patrick-tripp]

Hi Guy,

The links you tested are not the "regular grid" data but are the native ROMS grid. The URLs look very similar. The native ROMS output is being interpolated to a different grid. Can you please take another look using the following URL in ncWMS2.

https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/CBOFS/MODELS/2021/10/07/nos.cbofs.regulargrid.f001.20211007.t00z.nc

Here is the older wms page that displays the data fine: https://opendap.co-ops.nos.noaa.gov/thredds/godiva2/godiva2.html?server=https://opendap.co-ops.nos.noaa.gov/thredds/wms/NOAA/CBOFS/MODELS/2021/10/07/nos.cbofs.regulargrid.f001.20211007.t00z.nc

Thanks - Patrick

[guygriffiths]

Ok, with those I can reproduce the error.

Do you have a downloadable version of any of these files though? If I look them up in the opendap catalogue and do "Get Binary", I end up with a file which ncdump reports as being an unknown file format (and which ncWMS reports as "not a valid CDM file". Without a local copy of the data, it's particularly difficult to debug the problem.

[patrick-tripp]

CO-OPS is also running the fileServer where you can download them from. The data is only retained for two days, so you might have to change the date on the below link.

https://opendap.co-ops.nos.noaa.gov/thredds/fileServer/NOAA/CBOFS/MODELS/2021/10/15/nos.cbofs.regulargrid.f001.20211015.t00z.nc

Here is the TDS catalog page you can navigate from also: https://opendap.co-ops.nos.noaa.gov/thredds/catalog/NOAA/CBOFS/MODELS/catalog.html

I'm hoping to receive the source code that creates the regridded files soon. There might be something there that can be changed to resolve this.

Thanks

[guygriffiths]

@patrick-tripp Great, thanks for that. I've found the issue - all of the variables in that dataset have both grid and location attributes, which are usually used in staggered grids (http://sgrid.github.io/sgrid/). However, there is no staggered grid defined. But ncWMS2 is skipping those variables because it is assuming they have already been read by the staggered grid reading code. In short, it's a bug on our end, but it's easy to fix.

I'll fix it now and do a release shortly.

[guygriffiths]

Ok, this should be fixed in the latest version: https://github.com/Reading-eScience-Centre/ncwms/releases/tag/ncwms-2.5.1