euroargodev / argopy

A python library for Argo data beginners and experts
https://argopy.readthedocs.io
European Union Public License 1.2
176 stars 38 forks source link

Improve perf for long WMO list search with ArgoIndex #334

Open gmaze opened 6 months ago

gmaze commented 6 months ago

The modular method for search in ArgoIndex pyarrow and pandas backends is not an optimal design for large requests.

For instance the search_wmo methods:

are based on a simple loop through the list of WMOs to search. This could be optimised with apply methods for instance. But, in this cas could we keep the module design as it is ? (with a the construction of a binary mask that is then applied by the run method).

github-actions[bot] commented 3 months ago

This issue was marked as staled automatically because it has not seen any activity in 90 days