Closed JulesHuisman closed 2 years ago
@jlloyd-widen What do you think about this approach? I refactored the code so that all three approaches of configuration (upstream, hardcoded and dagster-configuration) are possible using the same class.
I still need to add some docstrings, and add tests for the different configuration methods.
There are now three ways of configuring the meltano elt solid.
@pipeline def meltano_pipeline(): MeltanoEltSolid( name="tap_csv_target_jsonl", tap="tap-csv", target="target-jsonl", job_id="csv-to-jsonl", tap_config={}, target_config={ "destination_path": "load" }, env_vars={"TAP_CSV__SELECT": json.dumps(["sample.id"])}, ).solid()