xhochy
commented
4 years ago There are several choices one can make:
- Does data fit into RAM?
- If not: memory-map to disk or distributed execution?
- Lazy or eager execution?
- Single vs multi-core
- CPU or GPU or something else?
- Inplace edits allowed or fully immutable?
- Numeric data, geographic data, string data?
Different answers will lead to different libraries or add-ons. I guess we don't have a working implementation for all answers but for a surprisingly large combination.
This though only gets you to the point to chose which DataFrame library, not where/how to store your data which is more #5.
https://pyviz.org gives an overview of the pyviz landscape, should we have something for the dataframes as well? How would one choose one library over the other, what are the strong and weak points of each library.