search

SpatioTemporal / STAREmaster

1 stars 3 forks source link

No cover information for MODIS 09 L2 files #202

Closed captainkirk99 closed 3 years ago

captainkirk99 commented 3 years ago

We have cover information in the MOD05 files, but not in the MOD09 files.

Yet I believe @jgallagher59701 is counting on cover information in these files.

I tried to adapt the MOD05 code to generate cover for the MOD09 files, but unfortunately the MOD09 files do not have the same GRing information in their structmetadata.0 attribute. So I'm not sure how to generate a cover.

Given that we have three resolutions of this file, 1km, 500m, and 250m, does that mean we can just use the 1 km data as the cover? @michaelleerilee is that correct?

captainkirk99 commented 3 years ago

Here's the structmetadata.0 attribute for MOD09:


GROUP=SwathStructure
    GROUP=SWATH_1
        SwathName="MODIS SWATH TYPE L2"
        GROUP=Dimension
            OBJECT=Dimension_1
                DimensionName="250m Data Lines"
                Size=8120
            END_OBJECT=Dimension_1
            OBJECT=Dimension_2
                DimensionName="250m Data Samples"
                Size=5416
            END_OBJECT=Dimension_2
            OBJECT=Dimension_3
                DimensionName="500m Data Lines"
                Size=4060
            END_OBJECT=Dimension_3
            OBJECT=Dimension_4
                DimensionName="500m Data Samples"
                Size=2708
            END_OBJECT=Dimension_4
            OBJECT=Dimension_5
                DimensionName="1km Data Lines"
                Size=2030
            END_OBJECT=Dimension_5
            OBJECT=Dimension_6
                DimensionName="1km Data Samples"
                Size=1354
            END_OBJECT=Dimension_6
        END_GROUP=Dimension
        GROUP=DimensionMap
            OBJECT=DimensionMap_1
                GeoDimension="1km Data Lines"
                DataDimension="500m Data Lines"
                Offset=0
                Increment=2
            END_OBJECT=DimensionMap_1
            OBJECT=DimensionMap_2
                GeoDimension="1km Data Samples"
                DataDimension="500m Data Samples"
                Offset=0
                Increment=2
            END_OBJECT=DimensionMap_2
            OBJECT=DimensionMap_3
                GeoDimension="1km Data Lines"
                DataDimension="250m Data Lines"
                Offset=1
                Increment=4
            END_OBJECT=DimensionMap_3
            OBJECT=DimensionMap_4
                GeoDimension="1km Data Samples"
                DataDimension="250m Data Samples"
                Offset=0
                Increment=4
            END_OBJECT=DimensionMap_4
        END_GROUP=DimensionMap
        GROUP=IndexDimensionMap
        END_GROUP=IndexDimensionMap
        GROUP=GeoField
            OBJECT=GeoField_1
                GeoFieldName="Latitude"
                DataType=DFNT_FLOAT32
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=GeoField_1
            OBJECT=GeoField_2
                GeoFieldName="Longitude"
                DataType=DFNT_FLOAT32
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=GeoField_2
        END_GROUP=GeoField
        GROUP=DataField
            OBJECT=DataField_1
                DataFieldName="1km Atmospheric Optical Depth Band 1"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_1
            OBJECT=DataField_2
                DataFieldName="1km Atmospheric Optical Depth Band 3"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_2
            OBJECT=DataField_3
                DataFieldName="1km Atmospheric Optical Depth Band 8"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_3
            OBJECT=DataField_4
                DataFieldName="1km Atmospheric Optical Depth Model"
                DataType=DFNT_UINT8
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_4
            OBJECT=DataField_5
                DataFieldName="1km water_vapor"
                DataType=DFNT_UINT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_5
            OBJECT=DataField_6
                DataFieldName="1km Atmospheric Optical Depth Band QA"
                DataType=DFNT_UINT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_6
            OBJECT=DataField_7
                DataFieldName="1km Atmospheric Optical Depth Band CM"
                DataType=DFNT_UINT8
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_7
            OBJECT=DataField_8
                DataFieldName="250m Surface Reflectance Band 1"
                DataType=DFNT_INT16
                DimList=("250m Data Lines","250m Data Samples")
            END_OBJECT=DataField_8
            OBJECT=DataField_9
                DataFieldName="250m Surface Reflectance Band 2"
                DataType=DFNT_INT16
                DimList=("250m Data Lines","250m Data Samples")
            END_OBJECT=DataField_9
            OBJECT=DataField_10
                DataFieldName="500m Surface Reflectance Band 1"
                DataType=DFNT_INT16
                DimList=("500m Data Lines","500m Data Samples")
            END_OBJECT=DataField_10
            OBJECT=DataField_11
                DataFieldName="500m Surface Reflectance Band 2"
                DataType=DFNT_INT16
                DimList=("500m Data Lines","500m Data Samples")
            END_OBJECT=DataField_11
            OBJECT=DataField_12
                DataFieldName="500m Surface Reflectance Band 3"
                DataType=DFNT_INT16
                DimList=("500m Data Lines","500m Data Samples")
            END_OBJECT=DataField_12
            OBJECT=DataField_13
                DataFieldName="500m Surface Reflectance Band 4"
                DataType=DFNT_INT16
                DimList=("500m Data Lines","500m Data Samples")
            END_OBJECT=DataField_13
            OBJECT=DataField_14
                DataFieldName="500m Surface Reflectance Band 5"
                DataType=DFNT_INT16
                DimList=("500m Data Lines","500m Data Samples")
            END_OBJECT=DataField_14
            OBJECT=DataField_15
                DataFieldName="500m Surface Reflectance Band 6"
                DataType=DFNT_INT16
                DimList=("500m Data Lines","500m Data Samples")
            END_OBJECT=DataField_15
            OBJECT=DataField_16
                DataFieldName="500m Surface Reflectance Band 7"
                DataType=DFNT_INT16
                DimList=("500m Data Lines","500m Data Samples")
            END_OBJECT=DataField_16
            OBJECT=DataField_17
                DataFieldName="1km Surface Reflectance Band 1"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_17
            OBJECT=DataField_18
                DataFieldName="1km Surface Reflectance Band 2"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_18
            OBJECT=DataField_19
                DataFieldName="1km Surface Reflectance Band 3"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_19
            OBJECT=DataField_20
                DataFieldName="1km Surface Reflectance Band 4"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_20
            OBJECT=DataField_21
                DataFieldName="1km Surface Reflectance Band 5"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_21
            OBJECT=DataField_22
                DataFieldName="1km Surface Reflectance Band 6"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_22
            OBJECT=DataField_23
                DataFieldName="1km Surface Reflectance Band 7"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_23
            OBJECT=DataField_24
                DataFieldName="1km Surface Reflectance Band 8"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_24
            OBJECT=DataField_25
                DataFieldName="1km Surface Reflectance Band 9"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_25
            OBJECT=DataField_26
                DataFieldName="1km Surface Reflectance Band 10"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_26
            OBJECT=DataField_27
                DataFieldName="1km Surface Reflectance Band 11"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_27
            OBJECT=DataField_28
                DataFieldName="1km Surface Reflectance Band 12"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_28
            OBJECT=DataField_29
                DataFieldName="1km Surface Reflectance Band 13"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_29
            OBJECT=DataField_30
                DataFieldName="1km Surface Reflectance Band 14"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_30
            OBJECT=DataField_31
                DataFieldName="1km Surface Reflectance Band 15"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_31
            OBJECT=DataField_32
                DataFieldName="1km Surface Reflectance Band 16"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_32
            OBJECT=DataField_33
                DataFieldName="BAND20"
                DataType=DFNT_UINT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_33
            OBJECT=DataField_34
                DataFieldName="1km Surface Reflectance Band 26"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_34
            OBJECT=DataField_35
                DataFieldName="BAND31"
                DataType=DFNT_UINT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_35
            OBJECT=DataField_36
                DataFieldName="BAND32"
                DataType=DFNT_UINT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_36
            OBJECT=DataField_37
                DataFieldName="BAND20ALBEDO"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_37
            OBJECT=DataField_38
                DataFieldName="250m Reflectance Band Quality"
                DataType=DFNT_UINT16
                DimList=("250m Data Lines","250m Data Samples")
            END_OBJECT=DataField_38
            OBJECT=DataField_39
                DataFieldName="500m Reflectance Band Quality"
                DataType=DFNT_UINT32
                DimList=("500m Data Lines","500m Data Samples")
            END_OBJECT=DataField_39
            OBJECT=DataField_40
                DataFieldName="1km Reflectance Band Quality"
                DataType=DFNT_UINT32
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_40
            OBJECT=DataField_41
                DataFieldName="1km b8-15 Reflectance Band Quality"
                DataType=DFNT_UINT32
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_41
            OBJECT=DataField_42
                DataFieldName="1km b16 Reflectance Band Quality"
                DataType=DFNT_UINT8
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_42
            OBJECT=DataField_43
                DataFieldName="1km Reflectance Data State QA"
                DataType=DFNT_UINT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_43
            OBJECT=DataField_44
                DataFieldName="1km Band 3 Path Radiance"
                DataType=DFNT_INT16
                DimList=("1km Data Lines","1km Data Samples")
            END_OBJECT=DataField_44
        END_GROUP=DataField
        GROUP=MergedFields
        END_GROUP=MergedFields
    END_GROUP=SWATH_1
END_GROUP=SwathStructure
GROUP=GridStructure
END_GROUP=GridStructure
GROUP=PointStructure
END_GROUP=PointStructure
END
NiklasPhabian commented 3 years ago

Here is why I pulled grings for MOD09 from:

from pyhdf.SD import SD

def get_metadata_group(hdf, group_name):
    metadata_group = {}
    keys = [s for s in hdf.attributes().keys() if group_name in s]
    for key in keys:    
        string = hdf.attributes()[key]
        m = parse_hdfeos_metadata(string)
        metadata_group  = {**metadata_group, **m}    
    return metadata_group

hdf = SD(file_path)    

core_metadata = get_metadata_group(hdf, 'CoreMetadata')    
g_points = core_metadata['INVENTORYMETADATA']['SPATIALDOMAINCONTAINER']['HORIZONTALSPATIALDOMAINCONTAINER']['GPOLYGON']['GPOLYGONCONTAINER']['GRINGPOINT']        
lats = g_points['GRINGPOINTLATITUDE']['VALUE']
lons = g_points['GRINGPOINTLONGITUDE']['VALUE']

gring_lats = list(map(float,lats.strip('()').split(', ')))[::-1]
gring_lons = list(map(float, lons.strip('()').split(', ')))[::-1]
NiklasPhabian commented 3 years ago

I guess in short; the GRING appears to be in the "CoreMetadata", not in the "StructureMetadata".

captainkirk99 commented 3 years ago

This is now working and MOD09L2 files are getting a cover. I will close this issue.