search

transferwise / pipelinewise-target-snowflake

Singer.io Target for Snowflake - PipelineWise compatible
https://transferwise.github.io/pipelinewise/
Other
51 stars 116 forks source link

Make data_retention_time_in_days configurable #331

Open deanmorin opened 1 year ago

deanmorin commented 1 year ago

Is your feature request related to a problem? Please describe. We have the DATA_RETENTION_TIME_IN_DAYS snowflake parameter set to 90 at the account level. However, when this target creates a table it's hardcoded to 1 day. This was discovered when we tried to use time travel to investigate a data issue and found we couldn't look back far enough.

Describe the solution you'd like It would be great for data_retention_time_in_days to be configurable. Ideally one of the options would be default where it wouldn't try to set it at all when creating the table, and it would just use the account/database/schema default.

Describe alternatives you've considered N/A

Additional context N/A

scottgoering commented 1 year ago

@deanmorin - any chance you want to submit your commit for PR? We just ran into the same issue and would love to see it merged in. Thanks!