Some NetCDF data cannot be read by `ucar.nc2.dataset.NetcdfDataset`

[anikfal]

Some NetCDF data (e.g. ERA5 atmospheric data) cannot be read by ucar.nc2.dataset.NetcdfDataset. However, some other files (e.g. VIIRS satellite data) can be read by NetcdfDataset without problem. Both files can be successfully read by ucar.nc2.NetcdfFile.

Regarding the error message, It seems to be a conversion issue. But how can I handle it? Is it about the file attributes that cannot be identified by NetcdfDataset?

Code:

import ucar.nc2.dataset.NetcdfDataset;

public class ReadNetcdf {
  public static void main(String[] args) {
    try {
      String filePath = "/home/anikfal/samples/era5.nc";
      NetcdfDataset ncDataset = NetcdfDataset.openDataset(filePath);
      System.out.println(ncDataset);
      ncDataset.close();
    } catch (Exception e) {
      System.out.println(e.getMessage());
    }
  }
}

Error message:

Exception in thread "main" java.lang.NoClassDefFoundError: ucar/units/ConversionException at ucar.nc2.dataset.conv.COARDSConvention.getAxisType(COARDSConvention.java:92) at ucar.nc2.dataset.conv.CSMConvention.getAxisType(CSMConvention.java:101) at ucar.nc2.dataset.conv.CF1Convention.getAxisType(CF1Convention.java:317) at ucar.nc2.dataset.CoordSysBuilder.makeCoordinateAxes(CoordSysBuilder.java:641) at ucar.nc2.dataset.CoordSysBuilder.buildCoordinateSystems(CoordSysBuilder.java:501) at ucar.nc2.dataset.NetcdfDataset.enhance(NetcdfDataset.java:469) at ucar.nc2.dataset.NetcdfDataset.(NetcdfDataset.java:1446) at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:427) at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:404) at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:389) at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:376) at ReadNetcdf.main(ReadNetcdf.java:7) Caused by: java.lang.ClassNotFoundException: ucar.units.ConversionException at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 12 more

[haileyajohnson]

Could you provide an example of a file that isn't working as a NetcdfDataset?

[anikfal]

@haileyajohnson Sure. The link below provides the file mentioned in the code: https://drive.google.com/file/d/1RjI8Mte5M9joROqVvHqR8yWGh-6fjdOa/view?usp=drive_link

[haileyajohnson]

I am unable to reproduce the issue with the file you've provided. Are you using the latest version of netCDF-Java?

Unrelated to your problem, but we recommend using the newer APIs to open netcdf files and datasets, e.g.: `NetcdfDataset dataset = NetcdfDatasets.openDataset(filepath);

[anikfal]

I came across a confusing situation:

I was not using the latest version (netcdfAll-5.5.3.jar). Instead, I was using the maven-built netcdf-4.3.22.jar.

With the latest version, the aforementioned problem was solved and era5.nc could be successfully read by NetcdfDataset. However, some other files that used to be read by netcdf-4.3.22.jar, could not be read by the latest version! I provide them below (they are standard and widely used environmental/atmospheric data): 1) VIIRS satellite data 2) WRF model output

More or less, the same situation happened for ucar.nc2.NetcdfFile.

---

I have a bunch of NetCDF files which have different metadata, attributes, and coordinate systems. This is probably something that causes such a problem and chaotic situation.

Can a user modify or develop the associated netcdf-java class and make it compatible with their own data and application?

[rschmunk]

@anikfal, I had no trouble opening any of your three examples. Boiled down, my code does

NetcdfFile njFile = NetcdfFiles.open ( filePathStr );
NetcdfDataset njdEnhanced  = NetcdfDatasets.enhance (njFile, enhSet, null);

where enhSet is my own enhancement set which omits one of the netCDF-Java defaults.

The VIIRS example, I do not see a lot-lat gridding system of any sort. Is it defined outside the dataset?

[anikfal]

NetcdfFile njFile = NetcdfFiles.open ( filePathStr );

I didn't know the new API has got such a syntax. Now I can open them all. Thanks.

The VIIRS example, I do not see a lot-lat gridding system of any sort. Is it defined outside the dataset?

Yes. Coordinates are stored in separate file.

[rschmunk]

@anikfal,

I didn't know the new API has got such a syntax.

There was some refactoring and adding of methods and classes for opening/acquiring datasets a few years ago, maybe when NJ 5.3 was in development. I'm not sure if the NetcdfFiles or NetcdfDatasets classes existed before then. I had to figure out how to use them because I was tired of getting deprecation alerts whenever I compiled my old code for opening datasets.