dbt-labs / dbt-external-tables

dbt macros to stage external sources
https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/
Apache License 2.0
286 stars 115 forks source link

[feature] additional metadata for snowpipe backed external tables #276

Closed dataders closed 2 months ago

dataders commented 3 months ago

Describe the feature

snowpipe-backed external tables should contain metadata$file_last_modified, whicn returns the last modified timestamp of the staged file the current row belongs to.

This metadata is useful for a use case where we'd like to filter staged files (from an S3 bucket) after a certain cutoff date, and we can't easily infer such date from the file name.