pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.08k stars 298 forks source link

Modis l1b doesn't check consistency between radiance and geolocation files #1834

Open ninahakansson opened 3 years ago

ninahakansson commented 3 years ago

Describe the bug The modis_l1b reader does not check that the geolocation file is from the same satellite or time as the corresponding radiance file.

To Reproduce

# Your code here
from satpy import Scene
from satpy.utils import debug_on
debug_on()
filename1 = "MYD021KM.A2010305.0025.061.2018062082014.hdf"
filename2 = "MOD03.A2010318.0025.061.2017256130255.hdf"

scn1 = Scene(reader='modis_l1b', filenames=[filename1])
scn1.load(['latitude', '1'])
print(scn1['latitude'][:].values) # Interpolated latitudes from 5km data

scn2 = Scene(reader='modis_l1b', filenames=[filename1, filename2])
scn2.load(['latitude', '1'])
print(scn2['latitude'][:].values) # Latitudes from Terra granule

*Expected behavior I expected an error or warning when using an Aqua radiance file with a Terra geolocation file (scn2). I expected the latitudes from the two scn obejcts to be the same. The Terra geolocation file (MOD03.A2010318.0025.061.2017256130255.hdf) should not be used to determine geolocation for the Aqua file MYD021KM.A2010305.0025.061.2018062082014.hdf.

Actual results Text output of actual results or error messages including full tracebacks if applicable.

The latitudes are not the same. The modis l1b reader reads channel data from the Aqua granule and geolocation data from the Terra granule (with different date) without complaining.

Screenshots If applicable, add screenshots to help explain your problem.

Environment Info:

Additional context Add any other context about the problem here.

djhoese commented 3 years ago

@ninahakansson Did this every work before? I made a lot of changes to the MODIS readers in the last version of Satpy, but I don't think this was ever something that was checked or supported.

The base reader classes in Satpy aren't really setup to do these types of comparisons between file types (MOD03 versus MOD02). They are loaded/opened separately.

Was specifying these two files at the same time something you want to work or was it an accident by you?

simonrp84 commented 3 years ago

No, to my knowledge this was a problem before as well. To be honest, I'm not sure this is something satpy should handle - there's so many potential cases where users could provide the wrong files or subtly different data than that which the reader expects, we can never catch all these possibilities.

mraspaud commented 3 years ago

I still think this is a bug somehow, at least the platform name should match...

simonrp84 commented 3 years ago

Actually yes that's a good point - I want to see what happens if you pass the reader both the MOD and MYD files for the same timestep...

djhoese commented 3 years ago

I'm not sure how the base reader class can make the assumption about this though. The metadata in the filename is not standardized (the reader can call the metadata whatever it wants), the platform_name property in the file handler is not standardized, and I don't think you can assume that all readers don't support mixed platforms. I'm sure there is some Level 2 or other algorithm that can produce and read files from different platforms... well maybe.

ninahakansson commented 3 years ago

@djhoese I run in to it by accident because I downloaded MOD03 instead of MYD03 files by mistake. @simonrp84 If I supply both MYD03 and MOD03 files both will be read and the latitude dataset will be twice the size of the radiance data.