Open luis-indicium opened 1 year ago
@luis-indicium this is an interesting idea!
How were you imagining this to be implemented? As a config value in your meltano.yml or maybe a path to a sql script?
If you provide a SQL script:
Without templating it might be difficult to use environments because table names will be hardcoded so you'll need different scripts for each environment.
Hi @pnadolny13
I think of the implementation in meltano.yml as a target config. We could pass a .sql file or a .sql description in the config itself.
Passing in the config, couldn't we define the stream that will be used through Meltano's pipelines?
Something similar to this:
https://hub.meltano.com/loaders/target-postgres--datamill-co/#before_run_sql-setting
Implement a Before_run_SQL for SQL commands before sending the data. This would make Drop_Table commands easier for projects that need it. Today doing this is not possible using Meltano.