The TableBackfill supports the creating of partitioned tables (by datasetKey), Apache Iceberg partitions supports the atomic update/replace of table partitions INSERT OVERWRITE TABLE occurrence PARTITION (dataset_key).
This feature can be used to keep the download table up-to-date and use the current table build process for disaster recovery and data synchronisation.
The TableBackfill supports the creating of partitioned tables (by datasetKey), Apache Iceberg partitions supports the atomic update/replace of table partitions
INSERT OVERWRITE TABLE occurrence PARTITION (dataset_key). This feature can be used to keep the download table up-to-date and use the current table build process for disaster recovery and data synchronisation.