dtcenter / MET

Model Evaluation Tools
https://dtcenter.org/community-code/model-evaluation-tools-met
Apache License 2.0
77 stars 24 forks source link

Enhance NetCDF support to read attributes as NC_STRING or NC_CHAR. #788

Closed dwfncar closed 7 years ago

dwfncar commented 7 years ago

This issue came up from a MET-Help ticket. NetCDF4 adds support for the NC_STRING data type. A user created a file for MET using NCL and the global and variable attributes were written using NC_STRING instead of NC_CHAR. This causes the MET to fail.


The task here is to update the logic of reading those attributes. Enhance the code to check the data type and allow them to be defined as NC_STRING or NC_CHAR (which is the current behavior).


Here's the MET-Help ticket:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=80092


And I've attached the sample file he sent. [MET-788] created by johnhg

dwfncar commented 7 years ago

From: "Roberto Garcia (INPE)" <roberto.garcia@inpe.br>


grid_stat FCT.nc OBS.nc GridStat.cfg -v 4


// Forecast fields to be verified ==============================
fcst = {
   field = [
      {
         name = "APCP";
         level = "(0,0,,)"; // 24-hour accumulation
         cat_thresh = [ NA ]; // [ >1.0, >=5.0 ] // ORIGINAL
      }
  ];
}
// Observation fields to be verified ==============================
obs = {
   field = [
      {
         name = "APCP";
         level = "(0,,)"; // 24-hour accumulation
         cat_thresh = [ NA ]; // [ >1.0, >=5.0 ] // ORIGINAL
      }
   ];
}


I'm getting the following error:


DEBUG 1: Default Config File:
/stornext/home/roberto.garcia/sfw/met-5.2/share/met/config/GridStatConfig_default
DEBUG 1: User Config File: GridStat.cfg
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
Met2dDataFile object of type "FileType_None".
ERROR :
ERROR : Trouble reading forecast file
"/stornext/grupos/pesq_dop/roberto.garcia/Eta40ens/Eta40ens_00_036hs.nc"
ERROR :

by hsoh

dwfncar commented 7 years ago

From: "John Halley Gotway" <johnhg@ucar.edu>


Run plot_data_plane version 6.0 to plot the file


met-6.0/bin/plot_data_plane FCT_edit.nc FCT_APCP.ps 'name="APCP";
level="(0,1,,)"; file_type=NETCDF_NCCF;'

by hsoh

dwfncar commented 7 years ago

The C++ API does not handle this well. The returned string value is very big.


The actual attributes are less than 80 bytes. API returned attribute which contains the proper attribute values but the size is too big:
     time_units: size: 20858744
 time_increment: size: 52608
 time_long_name: size: 19662160
  time_calendar: size: 0


This attribute size is not consistent. size of the second run:
     time_units: size: 32016248
 time_increment: size: 52608
 time_long_name: size: 30819664
  time_calendar: size: 0


This big size string causes seg-fault or "double free or corruption"


Error in `./plot_data_plane': double free or corruption (!prev): 0x00000000020c39a0

by hsoh

dwfncar commented 7 years ago

AOP 2017 VX1 NCAR by jensen

dwfncar commented 7 years ago

Changed error message. by hsoh