Delta Lake support open source connector

[parisni]

Hi delta lake provides ACID transformation to parquet files.

AWS provides a delta-lake connector for its commercial presto. Starbust blogged recently that they are in this process.

Is there any chance such connector will be accessible soon in presto ?

Thanks

[mahengyang]

I was use Delta Lake, but cannot join tables in hive and delta lake with spark sql, hop presto could support delta lake

[parisni]

@mahengyang : it is possible to create table in hive from delta. See https://github.com/delta-io/delta/issues/85#issuecomment-542111087

[BigDataDaddy]

Any movement in the last 8 months on this issue?

Databricks is claiming a Presto/Athena integration in public preview here: https://docs.databricks.com/delta/presto-integration.html with the Delta Lake open source version of the same here: https://docs.delta.io/latest/presto-integration.html

Connector code is here: https://github.com/delta-io/connectors

I see some serious flaws in the integration that make the HMS metadata for a table not compatible with other consumers of the HMS and for S3 a regression that eliminates EMRFS for EMR Presto re-exposing EMR Presto to S3 read after write eventual consistency issues.

FWIW, I'm posting a comment to this here in this repo because EMR Presto is still following and using this Presto fork.

[rohanpednekar]

cc @nmahadevuni

[karthik-balasubramanian]

@rohanpednekar , is there a timeline for supporting this feature?

[rohanpednekar]

https://github.com/prestodb/presto/pull/16843 is in progress to address this.

[dnskr]

@parisni The issue should be closed because Delta Lake connector is available since Presto 0.269