search

elixir-explorer / explorer

Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir
https://hexdocs.pm/explorer
MIT License
1.12k stars 123 forks source link

Support streaming: true on collect #941

Open josevalim opened 4 months ago

josevalim commented 4 months ago

https://github.com/pola-rs/polars/issues/3397#issuecomment-1341188319

spencerkent commented 4 months ago

Huge +1 for this. I was running into an issue the other day where a sequence of joins on lazy DataFrames was loading way more data than I thought it would into memory, and I'm thinking that streaming might help alleviate this issue.