CMR-6951 - switched muliti page requests from paging to scroll

nasa / eo-metadata-tools

eo-metadata-tools is a set of repositories for working with Earth Observation metadata. At its core are common libraries and demonstration scripts for accessing the Common Metadata Repository, to be accompanied by more specific modules and scripts to do dataset-specific queries, metadata validation, etc...

Apache License 2.0

25 stars 11 forks source link

CMR-6951 - switched muliti page requests from paging to scroll #12

Closed jceaser closed 3 years ago

jceaser commented 3 years ago

A very large change in the search api from page based requests to scrolling. Scrolling is auto created

Other changes include:

improved management of dictionary data
moved a huge chunk of common.search_by_page() to _make_search_request()
since the reason for moving to scroll was performance, also added experimintal_search_by_page_generator() to utilize python function generators
fixed logging and made it public to monitor scrolls.