chore: Make parquet reader options Comet options instead of Hadoop options

apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator

https://datafusion.apache.org/comet

Apache License 2.0

823 stars 163 forks source link

chore: Make parquet reader options Comet options instead of Hadoop options #968

Closed parthchandra closed 1 month ago

parthchandra commented 1 month ago

Which issue does this PR close?

Closes #876 .

Rationale for this change

Moves configuration options of the parallel reader into CometConf and adds documentation so that end users can easily understand the configuration of the reader.

How are these changes tested?

Existing unit test