search

apache / drill

Apache Drill is a distributed MPP query layer for self describing data
https://drill.apache.org/
Apache License 2.0
1.93k stars 980 forks source link

ANALYZE is not supported for group scan with format Delta #2809

Closed kmatt closed 1 year ago

kmatt commented 1 year ago

Describe the bug

Attempting to store table metadata in the Iceberg metastore, Delta tables which are queryable with SQL throw error when attempting ANALYZE TABLE

To Reproduce

apache drill (delta.root)> ANALYZE TABLE "path/to/table" REFRESH METADATA 'TABLE' LEVEL;

Error detail, log output or screenshots

Error: VALIDATION ERROR: ANALYZE is not supported for group scan [DeltaGroupScan [path="/path/to/table", entries=[ReadEntryWithPath [path=/path/to/table/partition/file.snappy.parquet], ...

Drill version

Apache Drill 1.21.1

Additional context

On same Drillbit, ANALYZE is successful on plain Parquet tables.

kmatt commented 1 year ago

https://issues.apache.org/jira/browse/DRILL-8440

vvysotskyi commented 1 year ago

It is not a bug, but a feature request. Currently, metastore doesn't support the Delta format plugin and the error message states that. In future, this functionality could be added.

kmatt commented 1 year ago

Closing as not an issue with this plugin.