microsoft / PlanetaryComputer

Issues, discussions, and information about the Microsoft Planetary Computer
https://planetarycomputer.microsoft.com/
MIT License
185 stars 8 forks source link

MODIS Fire products don't have platform label since day 2024065 #380

Open purkristin opened 1 month ago

purkristin commented 1 month ago

I'm querying products of MOD14A2 using platform in the query arguments. If I am filtering for "platform" = "terra" or "aqua" the latest item in 2024 is from day 065.

However if I don't filter for platform I do get the REAL latest item from the MODIS catalog. / at the time of writing this day 273

Example:

aoi = {
    "type": "Polygon",
    "coordinates": [
        [
            [21.56286942832358, 37.26398797975459],
            [25.65109227779675, 37.26398797975459],
            [25.65109227779675, 39.08464307335058],
            [21.56286942832358, 39.08464307335058],
            [21.56286942832358, 37.26398797975459],
        ]
    ],
}

search = catalog.search(
    collections="modis-14A2-061",
    intersects=aoi,
    datetime=f"2024-01-01/2024-12-31",
    query={"platform": {"eq": "aqua"}},
)

items = search.item_collection()

will produce 20 results

while the same search without the platform filter

search = catalog.search(
    collections="modis-14A2-061",
    intersects=aoi,
    datetime=f"2024-01-01/2024-12-31",
)

items = search.item_collection()

will produce 106 results (obviously also including terra at this point).

with the platform argument of the newer items being labeled as empty string.

latest_item = items[0]
latest_item.properties["platform"]
>>> ''
777arc commented 1 month ago

Looks like the XML from NASA stopped containing the Platform field, as of March 13 2024. Their XML used to contain:

  <Platforms>
    <Platform>
      <ShortName>Terra</ShortName>
      <Instruments>
        <Instrument>
          <ShortName>MODIS</ShortName>
        </Instrument>
      </Instruments>
    </Platform>
  </Platforms>

but for some reason they took it out. They also seemed to have removed TileID and they didn't add anything...

777arc commented 1 month ago

@purkristin if this is a serious problem for you then go ahead and open the issue in https://github.com/stactools-packages/modis so multiple groups can track it together. Thanks!

purkristin commented 1 month ago

Sure. But even without the metadata it is easy to tell if it's a terra or aqua product on the Modis side because terra is distributed as MOD and aqua as MYD. But in planetary computer both are distributed as MODxxx. so it would be nice if you could indicate in the platform label what the initial product was.

777arc commented 1 month ago

Ah, very good point, I'll get it added to our ingestion pipeline, TBD on when we'll be able to do the backfill

777arc commented 3 weeks ago

https://github.com/microsoft/planetary-computer-tasks/pull/315 (merge is waiting on a CICD fix)