search

z3z1ma / dbt-osmosis

Provides automated YAML management, a dbt server, streamlit workbench, and git-integrated dbt model output diff tools
https://z3z1ma.github.io/dbt-osmosis/
Apache License 2.0
422 stars 45 forks source link

Bug: Extra linebreak before injected column and missing linebreak after #40

Closed jaredx435k2d0 closed 1 year ago

jaredx435k2d0 commented 1 year ago

If I run dbt-osmosis yaml refactor on an existing source, and a column has been added to a table since the last time I ran it, it's added like this:

version: 2

sources:
  - name: my_source_name
    description: ""
    database: my_db_name
    schema: public
    tables:
      - name: my_table
        description: ""
        columns:
          - name: date
            description: ""

          - name: new_column_inserted_by_dbt_osmosis
      - name: my_next_table

Additional context python 3.9.16 macOS 13.2 (22D49) Snowflake 7.3.1 dbt-core==1.3.2 dbt-extractor==0.4.1 dbt-osmosis==0.9.8 dbt-snowflake==1.3.0

jaredx435k2d0 commented 1 year ago

Similarly, when removing an extra column from a yml file, the empty line separating tables will be removed:

Before:

models:
  - name: first_table
    description: ""
    columns:
      - name: id
      - name: extra_column_to_remove

  - name: second_table
    description: ""
    columns:
      - name: id
      - name: other_column

After:

models:
  - name: first_table
    description: ""
    columns:
      - name: id
  - name: second_table
    description: ""
    columns:
      - name: id
      - name: other_column
z3z1ma commented 1 year ago

This really boils down to the internals of ruamel.yaml which is the de/serialization library that manages the yaml. I am not sure how deeply I can tune that. I am going to close this as it is not an issue with the package but I am open to revisiting later as a feature/improvement request. Between now and then, I may have time to delve more into the knobs available in ruamel.yaml.