fraymio / modis-tools

Tools for working with the MODIS API and MODIS data.
Apache License 2.0
22 stars 11 forks source link

"updated" field not always available in ApiEntry (part of CollectionFeed) #3

Closed jonas-eberle closed 2 years ago

jonas-eberle commented 2 years ago

First of all thanks for this package. This looks great. Although it is currently focused on LPDAAC datasets, I think it can be easily adapated to datasets from NSIDC DAAC (e.g., MOD10A1 snow cover) as this dataaset is also available within CMR and download authentication is the same.

I ran into an issue while search for the MOD10A1 (version 061):

collections = collection_client.query(short_name="MOD10A1", version="061")

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/venv/lib/python3.8/site-packages/modis_tools/decorators.py", line 20, in wrapped
    result = func(*fargs, **fkwargs, **{reqarg_name: processed})
  File "/home/ubuntu/venv/lib/python3.8/site-packages/modis_tools/resources.py", line 32, in query
    collection_feed = CollectionFeed(**resp.json()["feed"])
  File "pydantic/main.py", line 331, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for CollectionFeed
entry -> 0 -> updated
  field required (type=value_error.missing)

You can have a look at the URL requested here: https://cmr.earthdata.nasa.gov/search/collections.json?doi=10.5067%2FMODIS%2FMOD10A1.061

This is related to the ApiEntry base model, which includes the updated property as a requirement: https://github.com/fraymio/modis-tools/blob/187a2519b4c5fa91a098b098e3eb5f9b8c2a16ef/modis_tools/models.py#L22-L31

It seems to be that this field is not always available. Would it be possible to declare this property as optional?

jamie-sgro commented 2 years ago

Hi Jonas, thanks for this very thorough issue report. I believe we can modify updated to be an optional argument. I'll speak with the team and make sure we can include this in the upcoming release