feat(data-lake): support automatic compaction of partitions

sam-goodwin / punchcard

Type-safe AWS infrastructure.

Apache License 2.0

507 stars 20 forks source link

feat(data-lake): support automatic compaction of partitions #52

Closed sam-goodwin closed 4 years ago

sam-goodwin commented 5 years ago

This change creates a second "hourly" table for all schemas in a DataLake. Minutely partitions are regularly compacted into hourly partitions with Step Functions and Lambda Functions..