search

dbt-labs / dbt-external-tables

dbt macros to stage external sources
https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/
Apache License 2.0
298 stars 120 forks source link

Query metadata of external tables #306

Open RalfKow opened 3 months ago

RalfKow commented 3 months ago

Describe the feature

There would be metadata columns added to the external table

Describe alternatives you've considered

Not using external tables and creating a table.

Additional context

I would love to use dbt external table but I need the file name because it contains valuable information that I need further down the pipelin

Who will this benefit?

Every use case that needs metadata

dataders commented 3 months ago

which data platform are you using?

RalfKow commented 3 months ago

I'm using Snowflake with external stages on Azure blobstorage

srjefers commented 2 months ago

If that is possible, I would like to be able to query metadata comming from the stage, METADATA$FILENAME or MEDATADATA$FILE_ROW_NUMBER, also is there a way that we can use custom column or that we can set a hardcoded value into the column? I mean something such like the current_date().