search

apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine
https://datafusion.apache.org/ballista
Apache License 2.0
1.4k stars 182 forks source link

refactor: port get_scan_files to Ballista #877

Closed alamb closed 9 months ago

alamb commented 10 months ago

Which issue does this PR close?

Related to https://github.com/apache/arrow-datafusion/issues/7357

Rationale for this change

We are sorting out how to split up DataFusion into smaller modules, and we hit https://github.com/apache/arrow-datafusion/issues/7357

There is a tension between knowing about datasources and separating out physical plan and datasources (see commentary on https://github.com/apache/arrow-datafusion/issues/7357 and linked PRs).

We thus propose to move this API upstream into Ballista. See https://github.com/apache/arrow-datafusion/pull/7487

What changes are included in this PR?

  1. Move this API upstream into Ballista: https://github.com/apache/arrow-datafusion/pull/7487

Are there any user-facing changes?

No

alamb commented 9 months ago

Thanks @Dandandan