search

apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
5.48k stars 1.01k forks source link

[Rust] DataFrame.collect should return RecordBatchReader #97

Open alamb opened 3 years ago

alamb commented 3 years ago

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-9763

DataFrame.collect should return RecordBatchReader instead of Vec in preparation for async support.

alamb commented 3 years ago

Comment from Andrew Lamb(alamb) @ 2021-04-26T12:33:20.013+0000:

Migrated to github: https://github.com/apache/arrow-rs/issues/95
houqp commented 2 years ago

I think this can be closed now since we already have an async collect method?

alamb commented 2 years ago

The async collect method is called DataFrame::execute_stream I believe: https://github.com/apache/arrow-datafusion/blob/58712078391502b740cf1f92f960edf4d1721187/datafusion/src/dataframe.rs#L269

@andygrove perhaps you have some insight on the intent of this issue (as you filed https://issues.apache.org/jira/browse/ARROW-9763 originally)