Error displaying/reading certain grib2 files on macOS

Dukestep commented 3 years ago

What is the bug or the crash?

On a Macbook Pro with Apple M1 Pro sillicon, QGIS cannot read some .grib2 files with gdal.

Seems like the reading of the file is fine (the metadata is properly read via gdal and via a gdalinfo) but QGIS reports the following error when dragging and dropping the file onto the canvas:

20211014T00Z_MSC_RDWPS-Atlantic-North-West_WVDIR_Sfc_RLatLon0.045_PT035H.grib2, band 1: IReadBlock failed at X offset 0, Y offset 0: Out of memory.

These files are displayed correctly with the same version of QGIS/GDAL on a Windows machine. I have not tested on an Intel-based Mac.

I believe this may be linked to the new Apple silicon but not entirely sure. Issue was present using both QGIS installed via homebrew and via the QGIS macOS installer.

Steps to reproduce the issue

Grab a samble grib2 file here: https://dd.weather.gc.ca/model_rdwps/atlantic-nw/5km/00/
Drag and drop the raster into QGIS via the Browser.
See error displayed in "!" next to layer name in the Layers panel.

Versions

QGIS version	3.22.0-Białowieża	QGIS code revision	d9022691f1
Qt version	5.15.2
Python version	3.9.5
GDAL/OGR version	3.3.2
PROJ version	8.1.1
EPSG Registry database version	v10.028 (2021-07-07)
GEOS version	3.9.1-CAPI-1.14.2
SQLite version	3.35.2
PDAL version	2.3.0
PostgreSQL client version	12.3
SpatiaLite version	5.0.1
QWT version	6.1.6
QScintilla2 version	2.11.5
OS version	macOS 12.0

Supported QGIS version

[X] I'm running a supported QGIS version according to the roadmap.

New profile

[X] I tried with a new QGIS profile

Additional context

No response

gioman commented 3 years ago

QGIS cannot read some .grib2 files with gdal.

@Dukestep can you gdalinfo/gdal_translate them from the command line?

Dukestep commented 3 years ago

QGIS cannot read some .grib2 files with gdal.

@Dukestep can you gdalinfo/gdal_translate them from the command line?

@gioman, yes I confirm that I can use both gdalinfo/gdal_translate via the command line (even using the binaries in /Applications/QGIS.app/Contents/MacOS/bin/!). That's why I decided to open the issue here rather than in the GDAL repository. gdalinfo -mm even calculates the minimum/maximum values which I assume means it can read the entire raster properly...?

gioman commented 3 years ago

20211014T00Z_MSC_RDWPS-Atlantic-North-West_WVDIR_Sfc_RLatLon0.045_PT035H.grib2, band 1: IReadBlock failed at X offset 0, Y offset 0: Out of memory.

@Dukestep opens fine on Win and Linux, so possibly a macOS only issue.

Dukestep commented 3 years ago

Yes, no issues detected on those OSes. Figure this may be linked to the arm64 arch of the new Apple silicon Macs and the emulation via Rosetta 2. Curious to see if those on Intel-based Macs are able to load the above grib2 files without any issues or if this can be replicated by other users running QGIS on Apple silicon.

Dukestep commented 3 years ago

I have update the issue title to reflect that a fellow user has reported the same error with the identical QGIS/GDAL versions on an Intel-based computer running macOS. This would seem to indicate that the problem affects macOS users in general.

bambryan commented 1 year ago

We are also having this same problem with M1 and M2 chip MacBooks with the latest versions of everything. Just wanted to upvote this.

conbrad commented 1 year ago

Also having the same issue on M1 Mac:

gdalinfo:

Driver: GRIB/GRIdded Binary (.grb, .grb2)
Files: CMC_glb_APCP_SFC_0_latlon.15x.15_2023071900_P045.grib2
Size is 2400, 1201
Coordinate System is:
GEOGCRS["Coordinate System imported from GRIB file",
    DATUM["unnamed",
        ELLIPSOID["Sphere",6371229,0,
            LENGTHUNIT["metre",1,
                ID["EPSG",9001]]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433,
            ID["EPSG",9122]]],
    CS[ellipsoidal,2],
        AXIS["latitude",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433,
                ID["EPSG",9122]]],
        AXIS["longitude",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433,
                ID["EPSG",9122]]]]
Data axis to CRS axis mapping: 2,1
Origin = (-180.074999999999989,90.075000000000003)
Pixel Size = (0.150000000000000,-0.150000000000000)
Corner Coordinates:
Upper Left  (-180.0750000,  90.0750000) (180d 4'30.00"W, 90d 4'30.00"N)
Lower Left  (-180.0750000, -90.0750000) (180d 4'30.00"W, 90d 4'30.00"S)
Upper Right ( 179.9250000,  90.0750000) (179d55'30.00"E, 90d 4'30.00"N)
Lower Right ( 179.9250000, -90.0750000) (179d55'30.00"E, 90d 4'30.00"S)
Center      (  -0.0750000,   0.0000000) (  0d 4'30.00"W,  0d 0' 0.00"N)
Band 1 Block=2400x1 Type=Float64, ColorInterp=Undefined
  Description = 0[-] SFC="Ground or water surface"
  Metadata:
    GRIB_COMMENT=45 hr Total precipitation [kg/(m^2)]
    GRIB_DISCIPLINE=0(Meteorological)
    GRIB_ELEMENT=APCP45
    GRIB_FORECAST_SECONDS=0
    GRIB_IDS=CENTER=54(Montreal) SUBCENTER=0 MASTER_TABLE=4 LOCAL_TABLE=0 SIGNF_REF_TIME=1(Start_of_Forecast) REF_TIME=2023-07-19T00:00:00Z PROD_STATUS=0(Operational) TYPE=2(Analysis_and_forecast)
    GRIB_PDS_PDTN=8
    GRIB_PDS_TEMPLATE_ASSEMBLED_VALUES=1 8 2 47 47 0 0 1 0 1 0 0 255 -127 -2147483647 2023 7 20 21 0 0 1 0 1 2 1 45 1 0
    GRIB_PDS_TEMPLATE_NUMBERS=1 8 2 47 47 0 0 0 1 0 0 0 0 1 0 0 0 0 0 255 255 255 255 255 255 7 231 7 20 21 0 0 1 0 0 0 0 1 2 1 0 0 0 45 1 0 0 0 0
    GRIB_REF_TIME=1689724800
    GRIB_SHORT_NAME=0-SFC
    GRIB_UNIT=[kg/(m^2)]
    GRIB_VALID_TIME=1689886800

QGIS Info:

dgboss commented 1 year ago

I'm seeing the same issue (IReadBlock failed at X offset 0, Y offset 0: Out of memory.) using QGIS v3.28.2 on a M1 Mac.

aminadibi commented 2 months ago

Same issue on M1 Mac with 16 GB of memory using QGIS version 3.34.11-Prizren

qgis / QGIS