z3z1ma / dbt-osmosis

Provides automated YAML management, a dbt server, streamlit workbench, and git-integrated dbt model output diff tools
https://z3z1ma.github.io/dbt-osmosis/
Apache License 2.0
422 stars 46 forks source link

[feature request] consistent YAML ordering #117

Open manypozo opened 7 months ago

manypozo commented 7 months ago

I find it useful to have consistent ordering on some sections of dbt, and default alphabetical ordering on some other sections.

The example below are sources, but the idea can be applied to other yaml documents too.

version: 2
sources:

# main keys are ordered consistently, where tables is the last key to ease file reading
- name: A
  description: 'description'
  database: 'DB'
  schema: 'SCHEMA'
  quoting:
   database: false
   schema: false
   identifier: false
  meta:
   key_1: 'a'
   key_2: 'b'

  freshness:
   warn_after: { count: 9999, period: day }
  loaded_at_field: updated_at::timestamp

  tags: []

  tables: # list of tables ordered Alphabetically
  - name: table_A
    description: 'description'
    columns:
    - name: col_a
      description: ' '        

    - name: col_b
      description: ' '     
      tests:

  - name: table_B
  - name: table_C

# if other sources in the file, then sources list is also ordered alphabetically
- name: B

Also, I would challenge the fact of ordering columns as the order coming from the DB. I would say alphabetical order makes reading easier.

I am not very familiar with dbt-osmosis code base yet, but happy to take a look at how could the be implemented if it is of any interest

z3z1ma commented 6 months ago

I think configurability here would make sense -- there is one place in the code base where we do the yaml dumping. I am fairly strongly of the opinion that DB ordering is better for a number of reasons. But I also think that folks will have differing opinions and needs to my own. So maybe a CLI flag is sufficient.