rsignell-usgs
commented
8 years ago @akirincich, I put this file on a THREDDS Data Server http://geoport-dev.whoi.edu/thredds/catalog/usgs/data2/rsignell/data/examples/catalog.html?dataset=usgs/data2/rsignell/data/examples/WHOI_ISLE_HFR_2014_8_1_800mgrid_1000mrad_16-Feb-2016.nc and then dropped the OPeNDAP Data URL into the IOOS Compliance Checker at http://data.ioos.us/compliance/index.html choosing the "CF 1.6 Compliance Check" from the drop down menu:
The report shows some missing units and coordinates attributes:
I then opened the file in the ToolsI application, and dumped the NcML. I made a few modifications, resulting in this, which is CF-Compliant:
<?xml version="1.0" encoding="UTF-8"?>
<ncml:netcdf xmlns:ncml="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" location="WHOI_ISLE_HFR_2014_2014_800mgrid_1000mrad_14-Feb-2016.nc">
<ncml:dimension name="t" length="100" />
<ncml:dimension name="lon" length="39" />
<ncml:dimension name="lat" length="36" />
<ncml:attribute name="CREATION_DATE" value="14-Feb-2016" />
<ncml:attribute name="creator_name" value="A. Kirincich" />
<ncml:attribute name="creator_email" value="akirincich@whoi.edu" />
<ncml:attribute name="id" value="hf_x_km" />
<ncml:attribute name="naming_authority" value="edu.whoi.mvco" />
<ncml:attribute name="title" value="HF Radar Data at MVCO" />
<ncml:attribute name="EXPERIMENT" value="Inner Shelf Lateral Exchange" />
<ncml:attribute name="summary" value="This data was collected by Kirincich as part of the Inner Shelf Lateral Exchange Study focused on examining the spatial variability of the mechanisms and process that lead to the exchange of water masses across the inner part of the continental shelf. The data consists of estimates of the near-surface horizontal (East and North) ocean currents made via High Frequency (HF) radar-based remote sensing of the ocean backscatter spectrum. The dataset spans the 3-month period of July, August, and September 2014 that overlapped both of the mass drifter releases conducted by Rypina and Kirincich. The effective measurement depth of the WHOI HF radars is 0.5 m below the ocean surface." />
<ncml:attribute name="DESCRIPTION" value="The WHOI HF radar system, as operated during the 2014 ISLE experiment, consisted of 3 land-based sites spaced at ~10km intervals along the south side of Martha’s Vineyard, MA. These 25MHz systems were run using a combination of 350kHz transmit bandwidth and low transmit power (10 W max) which allowed all systems to achieve resolutions of 429 m and ranges of 30 km (see Kirincich et al, 2016, for details). Received doppler spectra from each were processed using the advanced methods of Kirincich et al. (2012, JOTA) into radial velocity estimates every 15 min based on a 24 min averaging window. Radial velocity estimates were quality controlled before inclusion into the vector velocity estimates. using standard time-series QC techniques. These data were combined into vector velocities on a uniform 800m resolution grid, given in Latitude and Longitude coordinates, using a unique weighted least squares technique that employed non-velocity based signal quality metrics to weight the data to increase the accuracy of the final product. Two successive estimates of the 15 min radials are used to estimate the vector (east and north) velocities on a 1/2 hour time interval centered on the hour. The spatial extent of the vector velocities was limited by theoretical Geometrical Dilution of Precision (GDOP) values less than 1.75. An error estimate for the East, North, and total (norm) of the vector velocity components is given. This estimate uses the radial velocity error estimates (the weighted standard deviation of the individual HF radar radial returns found within each 5 degree azimuthal bin average) in a standard (Numerical Recipes) vector error calculation." />
<ncml:attribute name="DATA_ORIGIN" value="Woods Hole Oceanographic Institution, Department of Physical Oceanography" />
<ncml:attribute name="FUNDING_SOURCE" value="The observations used in this study were supported by NSF OCE Grant #1332626 and internal funding from the Woods Hole Oceanographic Institution." />
<ncml:attribute name="COORD_SYSTEM" value="GEOGRAPHIC" />
<ncml:attribute name="platform_type" value="High Frequency Radar" />
<ncml:attribute name="INST_TYPE" value="Codar Ocean Sensors Seasonde HF Radar" />
<ncml:attribute name="History" value="Converted to netCDF via MATLAB by A. Kirincich" />
<ncml:attribute name="min_latitude" type="double" value="41.08644" />
<ncml:attribute name="min_latitude_units" value="degree_north" />
<ncml:attribute name="max_latitude" type="double" value="41.33856" />
<ncml:attribute name="max_latitude_units" value="degree_north" />
<ncml:attribute name="min_longitude" type="double" value="-70.797912" />
<ncml:attribute name="min_longitude_units" value="degree_east" />
<ncml:attribute name="max_longitude" type="double" value="-70.435422" />
<ncml:attribute name="max_longitude_units" value="degree_east" />
<ncml:attribute name="Conventions" value="CF-1.6" />
<ncml:variable name="datetime" shape="t" type="double">
<ncml:attribute name="variable" value="Serial Date Number, GMT (Matlab convention)" />
<ncml:attribute name="units" value="days since 1970-01-01T00:00:00Z" />
<ncml:attribute name="calendar" value="gregorian" />
<ncml:attribute name="add_offset" type="double" value="-719529" />
<ncml:attribute name="note" value="A serial date number of 1 corresponds to Jan-1-0000" />
</ncml:variable>
<ncml:variable name="Longitude" shape="lon" type="double">
<ncml:attribute name="standard_name" value="longitude" />
<ncml:attribute name="variable" value="Gridded Longitude" />
<ncml:attribute name="units" value="degrees_east" />
</ncml:variable>
<ncml:variable name="Latitude" shape="lat" type="double">
<ncml:attribute name="standard_name" value="latitude" />
<ncml:attribute name="variable" value="Gridded Latitude" />
<ncml:attribute name="units" value="degrees_north" />
</ncml:variable>
<ncml:variable name="East_vel" shape="t lon lat" type="double">
<ncml:attribute name="coordinates" value="datetime Longitude Latitude" />
<ncml:attribute name="standard_name" value="barotropic_eastward_sea_water_velocity" />
<ncml:attribute name="variable" value="East_vel" />
<ncml:attribute name="units" value="cm/s" />
<ncml:attribute name="missing_value" type="double" value="999.0" />
</ncml:variable>
<ncml:variable name="North_vel" shape="t lon lat" type="double">
<ncml:attribute name="coordinates" value="datetime Longitude Latitude" />
<ncml:attribute name="standard_name" value="barotropic_northward_sea_water_velocity" />
<ncml:attribute name="variable" value="North_vel" />
<ncml:attribute name="units" value="cm/s" />
<ncml:attribute name="missing_value" type="double" value="999.0" />
</ncml:variable>
<ncml:variable name="East_err" shape="t lon lat" type="double">
<ncml:attribute name="coordinates" value="datetime Longitude Latitude" />
<ncml:attribute name="variable" value="East_err" />
<ncml:attribute name="units" value="cm2/s2" />
<ncml:attribute name="missing_value" type="double" value="999.0" />
</ncml:variable>
<ncml:variable name="North_err" shape="t lon lat" type="double">
<ncml:attribute name="coordinates" value="datetime Longitude Latitude" />
<ncml:attribute name="variable" value="North_err" />
<ncml:attribute name="units" value="cm2/s2" />
<ncml:attribute name="missing_value" type="double" value="999.0" />
</ncml:variable>
<ncml:variable name="err_cov" shape="t lon lat" type="double">
<ncml:attribute name="coordinates" value="datetime Longitude Latitude" />
<ncml:attribute name="variable" value="err_cov" />
<ncml:attribute name="units" value="cm2/s2" />
</ncml:variable>
<ncml:variable name="total_err" shape="t lon lat" type="double">
<ncml:attribute name="coordinates" value="datetime Longitude Latitude" />
<ncml:attribute name="variable" value="total_err" />
<ncml:attribute name="units" value="cm/s" />
<ncml:attribute name="missing_value" type="double" value="999.0" />
</ncml:variable>
</ncml:netcdf>
and this NcML dataset http://geoport-dev.whoi.edu/thredds/catalog/usgs/data2/rsignell/data/examples/catalog.html?dataset=usgs/data2/rsignell/data/examples/WHOI_ISLE_HFR_2014_2014_800mgrid_1000mrad_14-Feb-2016.ncml seems to work nicely.
Here's godiva2 showing the velocity field:
The main things that needed fixing were:
- time base should be in units since some date when the world agreed on the Gregorian Calendar (e.g.
days since 1900-01-01,seconds since 1970-01-01). In the NcML above, I did
<ncml:attribute name="units" value="days since 1970-01-01T00:00:00Z" />
<ncml:attribute name="calendar" value="gregorian" />
<ncml:attribute name="add_offset" type="double" value="-719529" />to convert to days since 1970.
- Changed to valid units for lon,lat
<ncml:variable name="Longitude" shape="lon" type="double">
<ncml:attribute name="standard_name" value="longitude" />
<ncml:attribute name="variable" value="Gridded Longitude" />
<ncml:attribute name="units" value="degrees_east" />
</ncml:variable>
<ncml:variable name="Latitude" shape="lat" type="double">
<ncml:attribute name="standard_name" value="latitude" />
<ncml:attribute name="variable" value="Gridded Latitude" />
<ncml:attribute name="units" value="degrees_north" />
</ncml:variable>- Added coordinates attribute and missing_value for the data variables:
<ncml:attribute name="coordinates" value="datetime Longitude Latitude" />
<ncml:attribute name="standard_name" value="barotropic_eastward_sea_water_velocity" />
<ncml:attribute name="variable" value="East_vel" />
<ncml:attribute name="units" value="cm/s" />
<ncml:attribute name="missing_value" type="double" value="999.0" />- Added
standard_nameattribute for variables where standard_name exists (full list at http://cfconventions.org/Data/cf-standard-names/29/build/cf-standard-name-table.html)
I'm new to CF-compliance issues and posting data online, but would like to learn how to make my HF radar data (remote sensing of the ocean surface currents via land-based radar systems) available to colleagues via standard, compliant servers. I've made an attempt to turn 1 day of data into a netcdf file from the native matlab formats. It should be available here:
https://drive.google.com/file/d/0B4Ftaivb4JbZUVpJUjhwUWZqaVk/view?usp=sharing
How close is this to 'cf-compliant' and how can I get it the rest of the way there, so that when I'm making new files, they can go right to a server.
Thanks, Anthony