Open BoPeng opened 5 years ago
Interesting talk at SciPy 2019, vaex acts on HDF5 files (that we use) but does the filtering and computation extremely fast with minimal memory usage (use memory map heavily).
The library is at https://github.com/vaexio/vaex
This could significantly improve the performance of our vtools update operation on statistics, and filtering operations for vtools associate.
vtools update
vtools associate
Interesting talk at SciPy 2019, vaex acts on HDF5 files (that we use) but does the filtering and computation extremely fast with minimal memory usage (use memory map heavily).
The library is at https://github.com/vaexio/vaex