z3z1ma / dbt-osmosis

Provides automated YAML management, a dbt server, streamlit workbench, and git-integrated dbt model output diff tools
https://z3z1ma.github.io/dbt-osmosis/
Apache License 2.0
422 stars 45 forks source link

Feature request: Case insensitivity when checking for columns #42

Closed jaredx435k2d0 closed 1 year ago

jaredx435k2d0 commented 1 year ago

Currently, if I have the following in _stg_my_schema.yml:

version: 2

models:
  - name: stg_my_schema__my_table
    description: ""
    columns:
      - name: id
      - name: _fivetran_synced

dbt-osmosis yaml refactor will remove _fivetran_synced and add _FIVETRAN_SYNCED, resulting in:

version: 2

models:
  - name: stg_my_schema__my_table
    description: ""
    columns:
      - name: id
      - name: _FIVETRAN_SYNCED

I'd like to request case insensitivity when looking for columns to add / remove.

I believe this applies more to DBs like Snowflake that display things in all caps by default (though the identifiers themselves are not actually in caps).

I'm definitely a Python rookie, but I tried changing c for c in yaml_file_model_section["columns"] if c["name"] != column to c for c in yaml_file_model_section["columns"] if c["name"].lower() != column.lower() in remove_columns_not_in_database()

and if column.lower() not in model_columns: to if column.upper() not in model_columns: in bootstrap_existing_model()

and it seemed to do the trick for me for now.

z3z1ma commented 1 year ago

This should now be resolved.