ioos / erddapy

Python interface for ERDDAP
https://ioos.github.io/erddapy/
BSD 3-Clause "New" or "Revised" License
77 stars 30 forks source link

Multiple ERDDAP search #190

Closed ocefpaf closed 3 years ago

ocefpaf commented 3 years ago

We could do an async search on a pre-determined list or maybe wrap an interface for [1].

@callumrollo is that something you would like to tackle?

[1] https://coastwatch.pfeg.noaa.gov/erddap/download/SearchMultipleERDDAPs.html

ocefpaf commented 3 years ago

Here is a hacky example on how to do that: https://nbviewer.jupyter.org/gist/ocefpaf/74f91d9aa9c4fa726bfba171b3edcadf

callumrollo commented 3 years ago

I tried integrating the extra checks into erddapy.url_handling._urlopen, but multiprocessing did not like the lru cache and kwargs, caused problems with the pickling. I think the least bad aproach would be a new function multi_urlopen for this specific purpose. I'll create a minimum working example