Blosc / python-blosc2

https://www.blosc.org/python-blosc2
Other
83 stars 19 forks source link

Filter and sort fields #324

Closed FrancescAlted closed 4 days ago

FrancescAlted commented 4 days ago

This allows performing data filtering, as well as sorting, in structured NDArrays. For example, given an array sarr with fields 'a', 'b' and 'c', the next:

farr = sarr["b >= c"].sort("c").indices().compute()

puts in farr the indices of the rows that fulfills that values in fields in 'b' are larger than values in 'c' ("b >= c" above), sorted by column 'c'.

farr is in turn an NDArray, so it is compressed; think of it as a compact index for other tasks. See the new examples/ndarray/filter_sort_fields.py self-contained script.