delta-io / kafka-delta-ingest

A highly efficient daemon for streaming data from Kafka into Delta Lake
Apache License 2.0
337 stars 71 forks source link

Preparing for deltalake v0.17 #162

Closed rtyler closed 5 months ago

rtyler commented 6 months ago

Lots of refactorings necessary, the dynamodb lock mechanism will be changing too!

rtyler commented 6 months ago

I am building other branches off of this one to help do integration testing and qualification of the next deltalake crate release