Closed martaiborra closed 2 years ago
We should also be able to work in parallel with views, which means we should be able to remove the following line: https://github.com/inaos/iron-array/blob/f71e29cbac9aa6ba8be2fc494dc631e62b6d868d/src/iarray_views.c#L573
If we would like to support the ITERBLOSC
eval method in slices, we would have to change how views are built. The schunk size would have to be the size from the view and not from the original viewed schunk. Furthermore, the chunk to be retrieved from the original container will have to be computed each time in the post filter, and in the worst scenario, we would have to read from 2 chunks when dim = 1
, 4 chunks when dim = 2
, 8 chunks when dim = 3
.... So instead, what we can do is support ITERBLOSC
in views only when the view's shape is the same than the viewed container's shape.
We did a benchmark to compare the old way of evaluating expressions with the new way, evaluating type views, and as expected, the evaluation is faster with the changes from PR #598 than with the old version.
Old version:
New version:
Currently, the expression evaluation defaults to use ITERCHUNK method whenever there is a view inside an expression (https://github.com/inaos/iron-array/blob/f71e29cbac9aa6ba8be2fc494dc631e62b6d868d/src/iarray_expression.c#L611). In order to get the same performance than iterblosc we need our views to be more flexible and accept this method as well.