search

C2SM / iconarray

R&D Python package for processing ICON data with xarray
https://c2sm.github.io/iconarray/
BSD 3-Clause "New" or "Revised" License
10 stars 2 forks source link

optimize crop #48

Closed cosunae closed 1 year ago

cosunae commented 1 year ago

Description

the cropping algorithm was slow, particularly noticeable for high resolutions like, R19B08. A performance bottleneck was here: https://github.com/C2SM/iconarray/blob/77b3d621067bc0bd27c9b4a682fb03bfcee73461/iconarray/core/crop.py#L243

where it filters out indices that are not in the list of selected indices (initially, based on coords within the cropping frame). But for each element the condition was checking whether the index is in a list, therefore O(N^2). In this PR we create a set out of the list, https://github.com/C2SM/iconarray/pull/48/files#diff-834d542c9c7b7e33f7e73e6ea15bbf8ea87e735868228c9816bd13db9bdeb33eR330 to be used in the check so that for each element the complexity is constant O(0), and therefore the entire selection is O(N)

cosunae commented 1 year ago

launch jenkins

cosunae commented 1 year ago

launch jenkins