KeyError: 'features' in search.py

[njdepsky]

Hi, I followed the first piece of your tutorial here exactly, installing the latest versions of intake-stac and sat-search. However, when I try and run get_STAC_items() for the AWS Sentinel-2 collection, I get a traceback error stemming from the satsearch/search.py function.

import intake
from satsearch import Search

def get_STAC_items(url, collection, dates, bbox):
    results = Search(url=url,
                        collections=[collection], 
                        datetime=dates,
                        bbox=bbox,    
                        sortby=['-properties.datetime'])

    items = results.items()
    print(f'Found {len(items)} Items')

    return intake.open_stac_item_collection(items)

url = 'https://earth-search.aws.element84.com/v0'
collection = 'sentinel-s2-l2a-cogs' # sentinel-2 w/ atmospheric corrections
bbox = [-122.4, 41.3, -122.1, 41.5] # (min lon, min lat, max lon, max lat)
dates = '2020-01-01/2020-12-31' 
items = get_STAC_items(url, collection, dates, bbox)

Error Message:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-6-4ce0732be3cb> in <module>
      3 bbox = [-122.4, 41.3, -122.1, 41.5] # (min lon, min lat, max lon, max lat)
      4 dates = '2020-01-01/2020-12-31'
----> 5 items = get_STAC_items(url, collection, dates, bbox)

<ipython-input-4-7bcb20e0751f> in get_STAC_items(url, collection, dates, bbox)
      6                         sortby=['-properties.datetime'])
      7 
----> 8     items = results.items()
      9     print(f'Found {len(items)} Items')
     10 

~/opt/anaconda3/lib/python3.8/site-packages/satsearch/search.py in items(self, limit, page_limit, headers)
    114                     _body.update(self.kwargs)
    115                 resp = self.query(url=nextlink['href'], headers=_headers, **_body)
--> 116             items += [Item(i) for i in resp['features']]
    117             links = [l for l in resp['links'] if l['rel'] == 'next']
    118             nextlink = links[0] if len(links) == 1 else None

KeyError: 'features'

I'm on MacOS Catalina 10.15.7 and the sat-search and intake-stac versions are both 0.3.0

[scottyhq]

@njdepsky your function isn't quite correct. You have results = Search( but need results = Search.search( so I'm guessing you typed it out rather than a copy paste from the blog post.

the syntax is a bit cumbersome currently due to the way this library is setup both as a CLI and library... (satsearch.Search.search)

[aliasmrchips]

This error also occurs when using the cli, presumably when no items are returned from a search.

[njdepsky]

Ok great thank you @scottyhq, silly mistake on my part! It looks to be working now, though now I am getting Request failed with status code 400 errors when trying to load the NASA Sentinel 1 collection shown in the blog. I noticed some other folks have been reporting similar issues in other issue threads here?

I double checked to make sure the username and password from my netrc file are correct and work for logging in to NASA EarthData in my browser.

My code is as follows:

import intake
from satsearch import Search
import netrc, fsspec, aiohttp

# NASA URS Authentication
(username, account, password) = netrc.netrc().authenticators("urs.earthdata.nasa.gov")
fsspec.config.conf['https'] = dict(client_kwargs={'auth': aiohttp.BasicAuth(username, password)})

def get_STAC_items(url, collection, dates, bbox):
    results = Search.search(url=url,
                        collections=[collection], 
                        datetime=dates,
                        bbox=bbox,    
                        sortby=['-properties.datetime'])

    items = results.items()
    print(f'Found {len(items)} Items')

    return intake.open_stac_item_collection(items)

# NASA - Sentinel-1 Data
url='https://cmr.earthdata.nasa.gov/stac/ASF'
bbox = [-122.4, 41.3, -122.1, 41.5] # (min lon, min lat, max lon, max lat)
collection = 'C1595422627-ASF' # sentinel-1 beta interferograms
dates = '2020-01-01/2020-12-31' 
items_s1 = get_STAC_items(url, collection, dates, bbox)

---------------------------------------------------------------------------
SatSearchError                            Traceback (most recent call last)
<ipython-input-15-08ead8f8ac7d> in <module>
      4 collection = 'C1595422627-ASF' # sentinel-1 beta interferograms
      5 dates = '2020-01-01/2020-12-31'
----> 6 items_s1 = get_STAC_items(url, collection, dates, bbox)

<ipython-input-13-1a5409101305> in get_STAC_items(url, collection, dates, bbox)
      6                         sortby=['-properties.datetime'])
      7 
----> 8     items = results.items()
      9     print(f'Found {len(items)} Items')
     10 

~/opt/anaconda3/lib/python3.8/site-packages/satsearch/search.py in items(self, limit, page_limit, headers)
     88     def items(self, limit=10000, page_limit=500, headers=None):
     89         """ Return all of the Items and Collections for this search """
---> 90         found = self.found(headers=headers)
     91         limit = self.limit or limit
     92         if found > limit:

~/opt/anaconda3/lib/python3.8/site-packages/satsearch/search.py in found(self, headers)
     60         url = urljoin(self.url, 'search')
     61 
---> 62         results = self.query(url=url, headers=headers, **kwargs)
     63         # TODO - check for status_code
     64         logger.debug(f"Found: {json.dumps(results)}")

~/opt/anaconda3/lib/python3.8/site-packages/satsearch/search.py in query(self, url, headers, **kwargs)
     78         # API error
     79         if response.status_code != 200:
---> 80             raise SatSearchError(response.text)
     81         return response.json()
     82 

SatSearchError: "Request failed with status code 400"

[scottyhq]

@njdepsky unfortunately the NASA CMR endpoint is currently incompatible with sat-search, it seems like a fix will be implemented soon according to https://github.com/sat-utils/sat-search/issues/106#issuecomment-784740852 . Note that the collection naming scheme might will be changing, so the code snippets in the blog post might also need adjusting

[njdepsky]

Ok great good to know, I'll keep my eye out, thanks!