search

ecmwf / cfgrib

A Python interface to map GRIB files to the NetCDF Common Data Model following the CF Convention using ecCodes
Apache License 2.0
407 stars 77 forks source link

Bug: Opening GRIB2 file with accompanying index file ignores index file. #328

Open JimBiardClimateAI opened 1 year ago

JimBiardClimateAI commented 1 year ago

I first open a GRIB2 file named d2m.grib with cfgrib.open_file(), which produces the index file d2m.grib.923a8.idx. In a different application I open the same GRIB2 file with cfgrib.open_dataset(). I receive the message Ignoring index file 'd2m.grib.923a8.idx' incompatible with GRIB file.

I am running with cfgrib 0.9.10.3 and xarray 2022.12.0 on MacOS Ventura 13.2.

JimBiardClimateAI commented 1 year ago

On deeper inspection, it appears that the issue is due to the index file having the absolute path to the GRIB2 file embedded within the file. I am moving the GRIB2 and .idx file pair to a new location before attempting to open the GRIB2 file. Is there any way that I can address this issue? I don't have the option of recreating the file path on the computer where I am using the files.

JimBiardClimateAI commented 1 year ago

Opening the GRIB2 file the first time with a relative path stores the relative path to the GRIB2 file in the .idx file, but the .idx file is still ignored if the current working directory is not set so that the relative path works. That is quite awkward for a large number of use cases.

Metamess commented 1 year ago

Hey Jim, I encountered your issue as well and made a larger write-up about the issue and potential ways to fix it (#350 ). Depending on the feedback I might try to write a PR to address it. If this is still relevant to you, you might be interested in any potential progress.