Problems with CF "Simple Geometry"

[JohnLCaron]

cdm/core/src/test/data/dataset/SimpleGeos/hru_soil_moist_vlen_3hru_5timestep.nc. (also in outflow_3seg_5timesteps_vlen.nc)

This is a netcdf-4 file with a variable length dimension, eg: double catchments_x(hruid=3, *); :axis = "X";

Open enhanced dataset so coordinate systems are added. Then try to read "catchments_x" coordinate, you get:

java.lang.ClassCastException: ucar.ma2.ArrayDouble$D1 cannot be cast to java.lang.Number

at ucar.nc2.dataset.EnhanceScaleMissingUnsignedImpl.convert(EnhanceScaleMissingUnsignedImpl.java:600)
at ucar.nc2.dataset.VariableDS.convert(VariableDS.java:246)
at ucar.nc2.dataset.VariableDS.convert(VariableDS.java:237)
at ucar.nc2.dataset.VariableDS._read(VariableDS.java:413)
at ucar.nc2.Variable.read(Variable.java:609)
at ucar.nc2.dataset.VariableDS.reallyRead(VariableDS.java:422)
at ucar.nc2.dataset.VariableDS._read(VariableDS.java:411)
at ucar.nc2.Variable.read(Variable.java:609)
at ucar.nc2.util.CompareNetcdf2.compareVariableData(CompareNetcdf2.java:508)
at ucar.nc2.util.CompareNetcdf2.compareVariables(CompareNetcdf2.java:296)
at ucar.nc2.util.CompareNetcdf2.compareVariable(CompareNetcdf2.java:268)
at ucar.nc2.util.CompareNetcdf2.compareCoordinateAxis(CompareNetcdf2.java:373)
at ucar.nc2.util.CompareNetcdf2.compareCoordinateSystem(CompareNetcdf2.java:354)
at ucar.nc2.util.CompareNetcdf2.compareVariables(CompareNetcdf2.java:336)
at ucar.nc2.util.CompareNetcdf2.compareGroups(CompareNetcdf2.java:241)
at ucar.nc2.util.CompareNetcdf2.compare(CompareNetcdf2.java:145)
at 

This happens at 5.0, would be interesting to know if it happens in 4.x.

Im guessing coordsys logic never tried to deal with a variable length coordinate ?

[JohnLCaron]

Added test that fails into TestSimpleGeom.

[JohnLCaron]

None of the files in /usr/local/google/home/jlcaron/github/repo2/netcdf-java/cdm/core/src/test/data/dataset/SimpleGeos

are parsed as a station feature type (via feature scan). Not sure whats up there, or what the "simple geometry" convention is. probably a seperate bug from this one, but will leave it bundled for now.

[lesserwhirls]

The CF Simple Geometry support (added to CF via cf-convention/cf-conventions#115) was a contribution from USGS (Unidata/thredds#1207). @cschroed-usgs, any insights into this particular issue?

[cschroedl-gov]

@lesserwhirls thanks for the shout-out. Nothing leaps to mind.

@JohnLCaron

Im guessing coordsys logic never tried to deal with a variable length coordinate ?

That's likely true.

Added test that fails into TestSimpleGeom.

Thanks! Can you please link to the test you mentioned?

None of the files in /usr/local/google/home/jlcaron/github/repo2/netcdf-java/cdm/core/src/test/data/dataset/SimpleGeos are parsed as a station feature type

I don't think the test files are intended to be parsed as station feature types. The CF Simple Geometries standard enables embedding non-gridded spatial entities (ex: watersheds, stream networks) inside NetCDF files. If there are station-like data in the test files, that is likely tangential.

[JohnLCaron]

Hi Carl;

The test is on a branch not yet checked in, but here is the code:

@Test
  public void testCoordinateVariables() throws IOException {
    String tstFile = TestDir.cdmLocalTestDataDir + "dataset/SimpleGeos/outflow_3seg_5timesteps_vlen.nc";
    // open the test file
    try (NetcdfDataset ncd = NetcdfDataset.openDataset(tstFile)) {
      for (CoordinateAxis axis : ncd.getCoordinateAxes()) {
        System.out.printf("Try to read %s ", axis.getFullName());
        Array data = axis.read();
        System.out.printf(" OK (%d) %n", data.getSize());
      }
    }
  }

[JohnLCaron]

There's a test file:

core/src/test/data/dataset/SimpleGeos/huc_helper_test.nc

  // global attributes:
  :Conventions = "CF-1.8";
  :featureType = "timeSeries";
  :cdm_data_type = "Station";
  :standard_name_vocabulary = "CF-1.7";
  :DODS.strlen = 12; // int
  :DODS.dimName = "name_strlen";

that claims its a station file. The CF simple geometry code lists station in a coordinate axis, but there is none:

int et(station=1, time=25);
  :units = "mm";
  :missing_value = -999; // int
  :long_name = "Area Weighted Mean Actual Evapotranspiration";
  :coordinates = "time lat lon";
  :geometry = "geometry_container";
  :grid_mapping = "crs";
  :geometry_type = "polygon";
  :node_count = "node_count";
  :node_coordinates = "x y";
  :part_node_count = "";
  :_CoordinateAxes = "time station x y";

it doesnt have a station coordinate, im thinking of removing it until someone can confirm its not malformed.

[cschroedl-gov]

Thanks for your help. My involvement in this contribution was pretty high-level, so I'm not clear on several details, including the precise intent of the test data. As far as I can tell, that NetCDF file is only used by one test class which is running this implementation: https://github.com/usgs/thredds/blob/74a45ae0abc6fcfa9a61f92f06bac85a4058861c/cdm/src/main/java/ucar/nc2/ft2/simpgeometry/CFSimpleGeometryHelper.java

The implementation doesn't appear to care about CDM data type. If dropping station stuff from the test data helps, I don't see much harm in that simplification.