QGIS not recognizing HDF4 on macOS

afitzgerrell commented 2 years ago

What is the bug or the crash?

When I add a file (MOD10A1.A2021346.h03v10.061.2021348044717.hdf) to a new project (from a fresh QGIS download/install) of QGIS at 3.16.15, I receive a warning that this "Invalid Data Source: is not a valid or recognized data source.". I'm working on a MacBook Pro. macOS Big Sur at v11.6.2.

Background: My prior version of QGIS at 3.10 A Coruña is compiled against GDAL/OGR | 2.4.1 and handles MODIS files well, but not HDF5 files. That prompted me to try upgrading to the latest LTR. While I think it's the GDAL version that's leading to A Coruña not accepting HDF5 files, I'm confused why Hannover, compiled against GDAL/OGR | 3.2.1, is not accepting HDF4 since my independent GDAL install (3.2.2) has drivers for both HDF4 and HDF5 files. Checking the GDAL website, and GDAL 3.2.1 should also contain drivers for HDF4.

I'll note that I don't think I'm doing anything incorrectly, but anything's possible! I apologize in advance if this is user-error as opposed to not an actual bug, but I'd appreciate learning what I should be doing differently if that's all it takes.

Many thanks! amy

Steps to reproduce the issue

Visit Earthdata Search and download the MOD10A1 v61 file MOD10A1.A2021346.h03v10.061.2021348044717.hdf. The following link should bring you to the granule to then download - you'll need to have an Earthdata Login to proceed with downloading the file, however (I'd have attached it here for you but that doesn't look to be allowed): https://search.earthdata.nasa.gov/search/granules?p=C1646610417-NSIDC_ECS&pg[0][v]=f&pg[0][id]=MOD10A1.A2021346.h03v10.061.2021348044717*&pg[0][gsk]=-start_date&g=G2183651300-NSIDC_ECS&q=mod10a1&tl=1640199283.717!3!!&m=-14.994140625000002!-150.626953125!4!1!0!0%2C2

Add this (either via dragging and dropping or via the Data Source Manager - Raster option) to a freshly installed, empty project, and the red message appears at the top of the application window.

Versions

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd">

Supported QGIS version

[X] I'm running a supported QGIS version according to the roadmap.

New profile

[X] I tried with a new QGIS profile

Additional context

No response

gioman commented 2 years ago

you'll need to have an Earthdata Login to proceed with downloading the file, however (I'd have attached it here for you but that doesn't look to be allowed)

@afitzgerrell it would be better if you could attach/link directly a file. Most of us won't have a Earthdata Login (I don't), thanks!

Are you on a Intel mac or a M1?

afitzgerrell commented 2 years ago

@gioman I'm on an intel mac.

When I try to attach a file, I'm informed "We don't support that type of file type". Is there another avenue available for me to attach a data (.hdf) file?

gioman commented 2 years ago

When I try to attach a file, I'm informed "We don't support that type of file type". Is there another avenue available for me to attach a data (.hdf) file?

@afitzgerrell you can attach ZIP files up to 10mb. If the ZIP is bigger use a Dropbox/Google Drive/etc share link.

afitzgerrell commented 2 years ago

Ah ha! Zip now attached. Many thanks!

MOD10A1.A2021346.h03v10.061.2021348044717.hdf.zip

gioman commented 2 years ago

Works on Linux and Windows, will try on macOS tomorrow.

afitzgerrell commented 2 years ago

Interesting, thanks @gioman. I have Windows running on a VM, so I downloaded and installed Hannover there. I see it does indeed work well on any type of HDF file (I don't normally work on that machine). I noticed the Windows-running Hannover is compiled with GDAL 3.4.0, but perhaps that's a red herring I'm focusing in on.

Looking forward to see how your testing with macOS goes tomorrow. Thanks!

Copied from the Windows VM Hannover "About QGIS" window: QGIS version 3.16.15-Hannover QGIS code revision e7fdad64 Compiled against Qt 5.15.2 Running against Qt 5.15.2 Compiled against GDAL/OGR 3.4.0 Running against GDAL/OGR 3.4.0 Compiled against GEOS 3.10.0-CAPI-1.16.0 Running against GEOS 3.10.0-CAPI-1.16.0 Compiled against SQLite 3.35.2 Running against SQLite 3.35.2 PostgreSQL Client Version 13.0 SpatiaLite Version 5.0.1 QWT Version 6.1.3 QScintilla2 Version 2.11.5 Compiled against PROJ 8.2.0 Running against PROJ Rel. 8.2.0, November 1st, 2021 OS Version Windows Server 2012 R2 Version 6.3 (Build 9600) Active python plugins db_manager; MetaSearch; processing

gioman commented 2 years ago

Looking forward to see how your testing with macOS goes tomorrow. Thanks!

This is gdalinfo --formats on Windows/Linux

  netCDF -raster,multidimensional raster,vector- (rw+s): Network Common Data Format
  HDF4 -raster,multidimensional raster- (ros): Hierarchical Data Format Release 4
  HDF4Image -raster- (rw+): HDF4 Dataset

On macOS it just returns

netCDF -raster,multidimensional raster,vector- (rw+s): Network Common Data Format

github-actions[bot] commented 2 years ago

The QGIS project highly values your report and would love to see it addressed. However, this issue has been left in feedback mode for the last 14 days and is being automatically marked as "stale". If you would like to continue with this issue, please provide any missing information or answer any open questions. If you could resolve the issue yourself meanwhile, please leave a note for future readers with the same problem and close the issue. In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this issue. If there is no further activity on this issue, it will be closed in a week.

afitzgerrell commented 2 years ago

Hi, Per the bot notice, I may have misunderstood or missed something, but I thought I was awaiting feedback on developer testing this issue in MacOS. I am quite interested in pursuing a fix for macOS if warranted, if possible.

gioman commented 2 years ago

I am quite interested in pursuing a fix for macOS if warranted, if possible.

@afitzgerrell see https://github.com/qgis/QGIS-Mac-Packager/issues/143#issuecomment-1010716669

qgis / QGIS-Mac-Packager