Handle Delft3D structured model output

[rsignell-usgs]

We should handle Delft3D structured model output, such as those described in #216 .

[rsignell-usgs]

I took the netcdf output from the matlab vs_trim2nc code that @drewstev ran and put the file on our geoport thredds server at: http://geoport-dev.whoi.edu/thredds/catalog/clay/usgs/users/rsignell/models/delft3d/catalog.html?dataset=clay/usgs/users/rsignell/models/delft3d/trim-mcr_riv_oc.nc With this "raw" output, the godiva2 viewer at http://geoport-dev.whoi.edu/thredds/godiva2/godiva2.html?server=http://geoport-dev.whoi.edu/thredds/wms/clay/usgs/users/rsignell/models/delft3d/trim-mcr_riv_oc.nc shows no variables to display, so the data is not really CF-compliant:

[rsignell-usgs]

Okay the good news is this file was easy to fix with NcML:

<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" location=
"trim-mcr_riv_oc.nc">

  <variable name="salinity">
    <attribute name="coordinates" value="time Layer latitude longitude"/>
    <attribute name="units" value="1"/>
  </variable>

  <variable name="velocity_x">
    <attribute name="coordinates" value="time Layer latitude longitude"/>
  </variable>

  <variable name="velocity_y">
    <attribute name="coordinates" value="time Layer latitude longitude"/>
  </variable>

  <variable name="depth">
    <remove type="attribute" name="positive"/>
    <attribute name="coordinates" value="latitude longitude"/>
  </variable>

  <variable name="waterlevel">
    <attribute name="coordinates" value="time latitude longitude"/>
    <remove type="attribute" name="positive"/>
  </variable>

</netcdf>

So if we go to http://geoport-dev.whoi.edu/thredds/catalog/clay/usgs/users/rsignell/models/delft3d/catalog.html?dataset=clay/usgs/users/rsignell/models/delft3d/00_dir.ncml and try the godiva2 link at http://geoport-dev.whoi.edu/thredds/godiva2/godiva2.html?server=http://geoport-dev.whoi.edu/thredds/wms/clay/usgs/users/rsignell/models/delft3d/00_dir.ncml we now see variables! Great!

The only problem is that Delft3D grids have missing_value in the lon,lat arrays, which is not strictly CF compliant, as described here: https://publicwiki.deltares.nl/display/NETCDF/Structured+grids#Structuredgrids-Missingcoordinates

So if we plot salinity using godiva2, it looks, uh, not so great:

[kwilcox]

The time values are really ugly here... not round at all. Is that a issue with the translation to NetCDF or is this just part of the simulation?

In [1]: import netCDF4 as nc4
In [2]: nc = nc4.Dataset("http://geoport-dev.whoi.edu/thredds/dodsC/clay/usgs/users/rsignell/models/delft3d/trim-mcr_riv_oc.nc")                                                             
In [3]: timevar = nc.variables['time']
In [4]: nc4.num2date(timevar[:], units=timevar.units)
Out[4]: 
array([datetime.datetime(2013, 5, 9, 0, 0, 3, 926754),
       datetime.datetime(2013, 5, 9, 1, 0, 3, 926811),
...
       datetime.datetime(2013, 5, 12, 21, 0, 3, 931743),
       datetime.datetime(2013, 5, 12, 22, 0, 3, 931800),
       datetime.datetime(2013, 5, 12, 23, 0, 3, 931847)], dtype=object)

[rsignell-usgs]

The good news is that NcWMS2/Godiva3, which will be in THREDDS 5.0, apparently works when there are missing values in lon,lat arrays. I manually put the OPeNDAP link into a NcWMS2 server, and then visualized the salinity field again with Godiva3: (choose "Delft3d-0001" from the list at http://geoport.whoi.edu/ncWMS2/Godiva3.html)

This means that when TDS5.0 comes out, we will be good to do with the ISO records straight from the TDS -- they will contain ncWMS2 endpoints which will just work.

And in the meantime, we can add Delft3D datasets manually to ncWMS2 and then modify the ISO records so they point to the ncWMS2 endpoint.

@kwilcox , how can we specify an appropriate WMS endpoint in the ISO metadata record for a dataset when the endpoint is coming from ncWMS2, which lists all datasets as layers in a single getcapabilities, like http://geoport.whoi.edu/ncWMS2/wms?SERVICE=WMS&REQUEST=GetCapabilities&VERSION=1.3.0

[kwilcox]

You could spin up a docker instance of ncWMS2 to just serve this model and use that GetCaps.

That being said, the styling in ncWMS2 is significantly different and you would just see an opacity slider in the Style panel in the portal.

[drewstev]

@kwilcox The strange timekeeping is in the native output files. It is strange, but not unique to these simulations...I have seen it in several other models that I have run. Is it a problem? If so I can modify the conversion code to round the time to the nearest XXX.

[kwilcox]

Rounding to the nearest minute in this case would pretty up the display and WMS GetCapabilities. Totally up to you though.