Unidata / thredds

THREDDS Data Server v4.6
https://www.unidata.ucar.edu/software/tds/v4.6/index.html
265 stars 179 forks source link

GRIB file causes NullPointerException. Missing table? #1017

Open ghost opened 6 years ago

ghost commented 6 years ago

I get a NullPointerException when trying to read this GRIB file (gziped so that GitHub will accept the upload):

W_fr-meteofrance,MODEL,ENSEMBLE+FORECAST+SURFACE+OLIVEPOLLEN+0H24H_C_LFPW_20180129000000.grib2.gz

I guess the issue is similar to #834 but in my case it seems template 40 definition is missing, as reported by the grib_dump program and by printing the template variable in Grib2Pds.java.

I'm looking into adding the template using other templates as examples. Overriding existing methods such as getGenProcessType is easy, but I'm not sure how to tackle fields that are not already there, such as Atmospheric Chemical Constituent Type. Should I create a new interface like PdsAerosol, PdsEnsemble, etc. ? Then should I register my new fields somewhere? If I look at how Grib2Pds48, I can see that its getAerosolType method gets called in Grib2CollectionPanel.java and gets added to PdsBeanInfo as aerType. Should I add a atmoChemConstType entry?

ghost commented 6 years ago

So i gave it a try: https://github.com/weatherforce/thredds/commit/51034e9b78f2fa5e7f4c0c91e06eb17b3a73f8a6 It seems to solve the problem. How does it look to you?

cofinoa commented 6 years ago

@amarandon

the change that you made to Grib2CollectionPanel.java only affects to the UI of the toolsUI application.

The constituentType=64002 value it's not defined at the GRIB2 table 4.230: http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-230.shtml

 # Analysis or forecast at a horizontal level or in a horizontal layer at a point in time for atmospheric chemical constituents (grib2/tables/5/4.0.table)  
  productDefinitionTemplateNumber = 40;
  # Atmospheric chemical or physical constituents (grib2/tables/5/4.1.0.table)  
  parameterCategory = 20;
  # Unknown code table entry (grib2/tables/5/4.2.0.20.table)  
  parameterNumber = 59;
  #-READ ONLY- parameterUnits = 59;
  #-READ ONLY- parameterName = 59;
  # Unknown code table entry (grib2/tables/5/4.230.table)  
  constituentType = 64002;
  # Forecast (grib2/tables/5/4.3.table)  
  typeOfGeneratingProcess = 2;
  generatingProcessIdentifier = 99;
  # Hour (grib2/tables/5/4.4.table)  
  indicatorOfUnitOfTimeRange = 1;
  # Hour (stepUnits.table)  

Do you have the GRIB2 local definitions for that constituents?

For the PDS template 40, the constituent at table 4.230 should be added to the variable Id and name. IMO, the place to do it is at ucar.nc2.grib.collection.Grib2Iosp extending the current naming variable algorithm to support chemical PDS (i.e. templates 40 and 48) and staellite sensors (templates 31 and 32). see http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclFormatSupport.shtml#GRIB2-support-details

@lesserwhirls what do you think?

ghost commented 6 years ago

@cofinoa Thank you for looking at this. No unfortunately I don't have GRIB2 local definitions for this constituent, although I know it's Olive Pollen in grain/m3.

cofinoa commented 6 years ago

@amarandon the only doc I'm able to find is this: http://www.regional.atmosphere.copernicus.eu/doc/Guide_Numerical_Data_CAMS_new.pdf

but I think it's missing the important information about how exactly the providers encoding data in GRIB2. Do you have access to whom is in charge?