search

delta-io / kafka-delta-ingest

A highly efficient daemon for streaming data from Kafka into Delta Lake
Apache License 2.0
359 stars 79 forks source link

What is the documentation to use HMS as the catalog with delta lake on S3? #174

Closed alberttwong closed 5 months ago

alberttwong commented 6 months ago

I can't seem to find instructions on how to write delta lake files on S3 or register them in HMS.

The closest I've found is using Apache xTable to covert iceberg or hudi on s3 into Delta Lake on s3 and then register those files in HMS. https://github.com/StarRocks/demo/tree/master/documentation-samples/datalakehouse

rtyler commented 5 months ago

This is a question better posted in the delta-users slack

alberttwong commented 5 months ago

My concern with delta-users slack is that this should be documented, and lots of stuff on stack isn't "official."

mightyshazam commented 5 months ago

My concern with delta-users slack is that this should be documented, and lots of stuff on stack isn't "official."

This is may be what you're looking for. However, the reason Tyler recommended the delta users slack is because this is definitely not the correct place for the question. https://github.com/delta-io/delta would be a better place to get the answer.