ESPRI-Mod / synda

ESGF Downloader (this is a deprecated repository, the tool has now moved to https://github.com/ESGF/esgf-download)
https://espri-mod.github.io/synda/
21 stars 11 forks source link

Optionally keep partial search results even after an exception #180

Open painter1 opened 3 years ago

painter1 commented 3 years ago

Add an option to save partial search results. That is, if a paginated search raises an exception, sdproxy.searchAPIProxy.call_web_service__PAGINATION will nevertheless return whatever incomplete results it has, rather than re-raise the exception.

This is very useful for production-scale replication where paginated searches are very large and failures not unknown. But by default this is turned off because a naive user would not be aware that the search was incomplete. To turn it on you have to set the constant NEVER_RAISE_EXCEPTION to True.