search

nctoolbox / nctoolbox

NCTOOLBOX A Matlab toolbox for working with common data model datasets
http://nctoolbox.github.io/nctoolbox
Other
128 stars 57 forks source link

nctoolbox support for ECMWF octahedral grib files #105

Open james-belanger opened 7 years ago

james-belanger commented 7 years ago

Currently using nctoolbox v1.1.3 ...

I have been unable to ingest ECMWF grib1 files into MATLAB using nctoolbox when the grib1 data is stored on ECMWF's octahedral grid. I can confirm that this is an error with nctoolbox ingest and not the data file as the eccodes utility grib_get_data is able to successfully output the data.

Attached is a sample data file ( echires_sfc_av_2016041500_2t_f000.grib.txt remove the .txt extension, I added it so that I could upload the data file) and the corresponding error output. 

>> in = ncgeodataset('echires_sfc_av_2016041500_2t_f000.grib');
2017-05-12 09:38:43,014 [main] WARN  ucar.nc2.grib.grib1.Grib1RecordScanner  - BAD GRIB-1 data message at 5212 header=  for=echires_sfc_av_2016041500_2t_f000.grib
2017-05-12 09:38:43,016 [main] WARN  ucar.nc2.grib.grib1.Grib1RecordScanner  - Bad Grib1 record in file echires_sfc_av_2016041500_2t_f000.grib, skipping pos=0
2017-05-12 09:38:43,051 [main] ERROR ucar.nc2.grib.GribCollection  - GribCollection echires_sfc_av_2016041500_2t_f000.grib: has no files
  made gbx9 index for echires_sfc_av_2016041500_2t_f000.grib size=40
  Index written: echires_sfc_av_2016041500_2t_f000.grib.gbx9
GribCollection echires_sfc_av_2016041500_2t_f000.grib: makeAggregatedGroups
 dcm= thredds.inventory.CollectionManagerSingleFile@536703f1
 Rectilyser2: nvars=0 records total=0 filtered=0 unique=0 dups=0 (NaN)
 createIndex for /home/jbelanger/github/gfs-research/Wind/ecmwf/echires_sfc_av_2016041500_2t_f000.grib.ncx
  write RecordMaps: bytes = 0 record = 0 bytesPerRecord=0

Error using ncdataset (line 97)
Failed to open echires_sfc_av_2016041500_2t_f000.grib

Error in cfdataset (line 59)
            obj = obj@ncdataset(url);

Error in ncgeodataset (line 74)
            obj = obj@cfdataset(url);

Caused by:
    Error using ncdataset (line 81)
    Java exception occurred:
    java.io.IOException: GribCollection echires_sfc_av_2016041500_2t_f000.grib has no files
        at ucar.nc2.grib.grib1.Grib1CollectionBuilder.createIndex(Grib1CollectionBuilder.java:580)
        at ucar.nc2.grib.grib1.Grib1CollectionBuilder.createIndex(Grib1CollectionBuilder.java:422)
        at ucar.nc2.grib.grib1.Grib1CollectionBuilder.readOrCreateIndex(Grib1CollectionBuilder.java:157)
        at ucar.nc2.grib.grib1.Grib1CollectionBuilder.readOrCreateIndexFromSingleFile(Grib1CollectionBuilder.java:68)
        at ucar.nc2.grib.GribIndex.makeGribCollectionFromSingleFile(GribIndex.java:121)
        at ucar.nc2.grib.grib1.Grib1Iosp.open(Grib1Iosp.java:135)
        at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:1499)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:887)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:496)
        at ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:693)
        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:424)
        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:407)
        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:392)
        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:379)
hohonuuli commented 7 years ago

Yep, I can confirm that error. Looks like its a problem in the grib library.

>> nc = ncdataset('/Users/brian/Downloads/echires_sfc_av_2016041500_2t_f000.grib')
2017-05-15 08:28:06,483 [main] WARN  ucar.nc2.grib.grib1.Grib1RecordScanner  - BAD GRIB-1 data message at 5212 header=  for=/Users/brian/Downloads/echires_sfc_av_2016041500_2t_f000.grib
2017-05-15 08:28:06,485 [main] WARN  ucar.nc2.grib.grib1.Grib1RecordScanner  - Bad Grib1 record in file /Users/brian/Downloads/echires_sfc_av_2016041500_2t_f000.grib, skipping pos=0
2017-05-15 08:28:06,524 [main] ERROR ucar.nc2.grib.grib1.Grib1Iosp  - GribCollection echires_sfc_av_2016041500_2t_f000.grib: has no files
Error using ncdataset (line 97)
Failed to open /Users/brian/Downloads/echires_sfc_av_2016041500_2t_f000.grib

Caused by:
    Error using ncdataset (line 81)
    Java exception occurred:
    java.io.IOException: java.io.IOException: GribCollection echires_sfc_av_2016041500_2t_f000.grib has no files
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:425)
        at ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:699)
        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:421)
        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:404)
        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:389)
        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:376)
    Caused by: java.io.IOException: GribCollection echires_sfc_av_2016041500_2t_f000.grib has no files
        at ucar.nc2.grib.grib1.Grib1CollectionBuilder.createIndex(Grib1CollectionBuilder.java:608)
        at ucar.nc2.grib.grib1.Grib1CollectionBuilder.createIndex(Grib1CollectionBuilder.java:446)
        at ucar.nc2.grib.grib1.Grib1CollectionBuilder.readOrCreateIndex(Grib1CollectionBuilder.java:151)
        at ucar.nc2.grib.grib1.Grib1CollectionBuilder.readOrCreateIndexFromSingleFile(Grib1CollectionBuilder.java:66)
        at ucar.nc2.grib.GribIndex.makeGribCollectionFromSingleFile(GribIndex.java:120)
        at ucar.nc2.grib.grib1.Grib1Iosp.open(Grib1Iosp.java:136)
        at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:1521)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:813)
        at ucar.nc2.NetcdfFile.open(NetcdfFile.java:422)
        ... 5 more