Use print for outputting codegen

JorgenG commented 2 years ago

This is a:

[x] bug fix PR with no breaking changes — please ensure the base branch is main
[ ] new functionality — please ensure the base branch is the latest dev/ branch
[ ] a breaking change — please ensure the base branch is the latest dev/ branch

Description & motivation

This PR switches from using the log function to using the print function. The main motivation for doing this is to enable piping the output from a codegen run operation command directly into the target file, instead of copy pasting.

Example:

dbt --quiet run-operation generate_model_yaml --args '{"model_name": "stg_jaffle_shop__orders"}' > models/staging/jaffle_shop/stg_jaffle_shop__orders.yml

This would also enable further automation, i.e. generating model yaml for all files in a directory etc.

Should resolve https://github.com/dbt-labs/dbt-codegen/issues/56 and https://github.com/dbt-labs/dbt-codegen/issues/71

Checklist

[x] I have verified that these changes work locally
[x] I have updated the README.md (if applicable)
[x] Not applicable - I have added tests & descriptions to my models (and macros if applicable)
[x] I have added an entry to CHANGELOG.md

swiatek25 commented 1 year ago

@JorgenG why have you closed that PR? Is there any other way to achieve what print was supposed to deliver?

JorgenG commented 1 year ago

@swiatek25 It seemed to be no interest in it and we have already skipped using the package and added our own implementation of this. The approach is valid, but niche. Outputting to file has it's challenges. Regardless, I'll reopen the PR if there seems to be interest in it.

jenna-jordan commented 1 year ago

Hello,

I have been struggling for days now to figure out how to output to a file instead of the standard output, and finally started investigating the codegen repo and found this PR. I would like to say that I am very interested in this feature, at the moment it is impossible to pipe output into a file (below is an example of what gets piped to the file instead):

[0m17:40:24  Running with dbt=1.3.1
[0m17:40:25  Encountered an error:
Parsing Error
  Error reading analytics: internal\stg\test.yml - Runtime Error
    unacceptable character #x001b: control characters are not allowed
      in "<unicode string>", position 0

So I just wanted to contribute my opinion that this would be a very desirable feature as the capability of outputting to a file would remove a lot of necessary manual effort when using codegen.

swiatek25 commented 1 year ago

@jenna-jordan I worked around it by creating a wrapper macro that simply prints returned value from codegen macro.

{%- macro generate_model_metadata(model_name) -%}
    {% set model_yml = codegen.generate_model_yaml(model_names=[model_name]) %}
    {{ print(model_yml) }}
{%- endmacro %}

That is surely not perfect but considering I use a small subset of codegen macros it is acceptable.

Nevertheless I'm still voting for this PR.

jenna-jordan commented 1 year ago

@swiatek25 how would you call this macro as a dbt run operation within a shell script?

swiatek25 commented 1 year ago

@swiatek25 how would you call this macro as a dbt run operation within a shell script?

dbt --quiet run-operation generate_model_metadata --args '{"model_name": "stg_jaffle_shop__orders"}' > output.file

note --quiet flag

jaypeedevlin commented 11 months ago

@joellabes are you still looking after this repo, this seems like a good QOL addition (folks seem to be working around it right now).

Also https://github.com/dbt-labs/dbt-codegen/blob/main/dbt_project.yml#L4 needs to be bumped to >=1.1.0 to get the print feature in.

dbt-labs / dbt-codegen

Use print for outputting codegen #86

Description & motivation

Checklist