dtcenter / MET

Model Evaluation Tools
https://dtcenter.org/community-code/model-evaluation-tools-met
Apache License 2.0
74 stars 22 forks source link

pb2nc does not handle more than 255 vertical levels #852

Closed dwfncar closed 6 years ago

dwfncar commented 6 years ago

pb2nc does not generate NetCDF output (segmentation fault) if the input BUFR file has more than 255 vertical levels. For example, nam.t00z.radwnd.tm02.bufr_d has 500 vertical levels.


The code was designed to interact between C and Fortran with the pre-allocated static buffer. Instead of using dynamic buffer, it will ignore the vertical levels above the pre-defined maximum vertical level and give the warning.


The maximum vertical level will be increased to 1023. [MET-852] created by hsoh

dwfncar commented 6 years ago

The execution time was increased about 4 times longer with the same PREPBUFR file (less than 250 levels)


Here is a simple test with handling different vertical levels with the same input PREPBUFR (the input has less than 255 vertical levels):


From nightly build log file (NB20170808.out):
Before (rev 4771)
NB20170808.out:TEST: pb2nc_GDAS_mask_grid_G212- pass - 345.791 sec
NB20170808.out:TEST: pb2nc_NDAS_mask_poly_conus - pass - 69.503 sec


After (rev 4784):
NB20170808.out:TEST: pb2nc_GDAS_mask_grid_G212- pass - 1320.358 sec
NB20170808.out:TEST: pb2nc_NDAS_mask_poly_conus - pass - 252.191 sec

by hsoh

dwfncar commented 6 years ago

Added test case (pb2nc_vertical_level_500).


Modify logic of setting the buffer size (discussed with John).
Check contents of obs_bufrvar and ignore the standard 6 variables and the derived variables beginning with "D". If 0 vars are left, use the small buffer. If 1 or more vars are left, use the big buffer.


The 255 vertical levels will be processes for following cases:

by hsoh