quantile-development / dagster-meltano

A Dagster plugin that allows you to run Meltano in Dagster
MIT License
41 stars 17 forks source link

draft: Strip equal sign in generate_dagster_name() and add possibillity of custom dagster_name #45

Closed ReneTC closed 9 months ago

ReneTC commented 9 months ago

Adding .replace("=", "_") the the generate_dagster_name function. As Meltano prod runs are called with:

meltano --environment=prod run your:command

Draft: As I was not able to test this

JulesHuisman commented 9 months ago

To test it you can either create a pytest test.

Or you can use this project to test it: https://github.com/quantile-development/dagster-meltano/tree/master/meltano_project

JulesHuisman commented 9 months ago

And if you could bump the version here: https://github.com/quantile-development/dagster-meltano/blob/master/pyproject.toml

That way it's easier to publish for me :)

ReneTC commented 9 months ago

To test it you can either create a pytest test.

Or you can use this project to test it: https://github.com/quantile-development/dagster-meltano/tree/master/meltano_project

As in independent function it works as expected. I wanted originally to test it with dagster.

And if you could bump the version here: https://github.com/quantile-development/dagster-meltano/blob/mast/pyproject.toml That way it's easier to publish for me :)

I bumped it back actually as I couldn't get my conflict to work. So the only changed file in this MR is dagster_meltano/utils.py

ReneTC commented 9 months ago

Added posibility of a custom dagster_name. It is just a suggestion, feel free to suggest changes or revert.

ReneTC commented 9 months ago

Getting messy, I'll close this and only keep the string replace =