search

brazil-data-cube / stac.py

Python Client Library for SpatioTemporal Asset Catalog
MIT License
5 stars 16 forks source link

TypeError for list comprehension for Summaries and Providers #104

Closed Mahir-Sparkess closed 3 years ago

Mahir-Sparkess commented 3 years ago

When running the service.catalog method, it would raise a TypeError, the console log shows this:

client = STAC(url=url, verify=False)
Debugger connected.
client.catalog
/Users/kazi.mahir/python-virtual-environments/stac/lib/python3.9/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host 'stac-elasticsearch-master.130.246.131.9.nip.io'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(
/Users/kazi.mahir/python-virtual-environments/stac/lib/python3.9/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host 'stac-elasticsearch-master.130.246.131.9.nip.io'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(
Traceback (most recent call last):
  File "/Users/kazi.mahir/python-virtual-environments/stac/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3441, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-5-29f2f637bd2d>", line 1, in <module>
    client.catalog
  File "/Users/kazi.mahir/Documents/git/stac.py/stac/stac.py", line 64, in catalog
    self.collections
  File "/Users/kazi.mahir/Documents/git/stac.py/stac/stac.py", line 85, in collections
    self._collections = {collection['id']: Collection(collection, self._validate, **self._request_kwargs)
  File "/Users/kazi.mahir/Documents/git/stac.py/stac/stac.py", line 85, in <dictcomp>
    self._collections = {collection['id']: Collection(collection, self._validate, **self._request_kwargs)
  File "/Users/kazi.mahir/Documents/git/stac.py/stac/collection.py", line 113, in __init__
    self._providers = [Provider(provider) for provider in self['providers']] if 'providers' in self else []
TypeError: 'NoneType' object is not iterable

I am unsure why I would get a TypeError since my providers' are null in the JSON, so it should skip the statement and return an empty list, same with my summaries. My solution: if it crashes at a TypeError, then catch the error and return the empty dict or list, as if to skip the statement.

Mahir-Sparkess commented 3 years ago

Hello again, I have found the cause of my TypeError. In my STAC collection, we have 'providers' and 'summaries' stated even if they are a None value:

'summaries': {'mip_era': ['CMIP6'], 'activity_id': ['AerChemMIP'], 'source_id': ['CNRM-ESM2-1'], 'table_id': ['Amon', 'Lmon', 'Omon'], 'institution_id': ['CNRM-CERFACS']},
'providers': Null

Due to our STAC implementation explicitly states 'proivders' even if we have None, it passes the if statement for the Collection class. Which is then passing a None value into the list comprehension that is causing my error.

So instead in my branch, I've changed the if statement from: https://github.com/brazil-data-cube/stac.py/blob/b52ed35aee07dad0c4e4588722b7234cb28b44b4/stac/collection.py#L112-L113 Replacing the in to a get which also fixes my issue: https://github.com/cedadev/stac.py/blob/2ddaae900c0968e192c7173c3152b699afc04ed7/stac/collection.py#L111-L112

I've also updated my PR #105 so that it's no longer using a try/except statement anymore. I hope this is acceptable :)