search

apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
5.87k stars 1.11k forks source link

Read encrypted PARQUET files #5763

Open jaimesalvador opened 1 year ago

jaimesalvador commented 1 year ago

Is your feature request related to a problem or challenge?

I'm trying to use DATAFUSION to load some parquet-encrypted files, but I can't find the option to set the encryption parameters (key, etc).

I found ParquetReadOptions, but it seems don't have a encryption options.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

tustvold commented 1 year ago

https://github.com/apache/arrow-rs/issues/3511

leetcode-1533 commented 5 months ago

I am interested and would like to give it a try.

alamb commented 4 months ago

FWIW there is a parquet reader that supports encryption in comet that might serve as some inspiration / source of code https://github.com/apache/datafusion-comet/tree/main/core/src/parquet