Ordering of custom config stanzas in default.yml

[BertieW]

Ability to define custom configurations via the default.yml is one of my favourite features of the install script, however, its use for defining indexes.conf is limited somewhat due to the fact that stanza order isn't preserved when yaml is imported.

Specifically, default.yml snippet written like so:

indexes:
      directory: /opt/splunk/etc/system/local/
      content:
        default:
          remotePath: "volume:remote_store/$_index_name"
          repFactor: auto
        "volume:remote_store":
          storageType: remote
          path: "s3://bucket-name"
          remote.s3.encryption: sse-s3
        alerts:
          homePath: $SPLUNK_DB/alerts/db
          thawedPath: $SPLUNK_DB/alerts/thaweddb
          coldPath: $SPLUNK_DB/alerts/colddb
        cm_events:
          homePath: $SPLUNK_DB/cm_events/db
          thawedPath: $SPLUNK_DB/cm_events/thaweddb
          coldPath: $SPLUNK_DB/cm_events/colddb
        cm_metrics:
          homePath: $SPLUNK_DB/cm_metrics/db
          thawedPath: $SPLUNK_DB/cm_metrics/thaweddb
          coldPath: $SPLUNK_DB/cm_metrics/colddb
          datatype: metric
        cm_meta:
          homePath: $SPLUNK_DB/cm_meta/db
          thawedPath: $SPLUNK_DB/cm_meta/thaweddb
          coldPath: $SPLUNK_DB/cm_meta/colddb

is rendered like so:

indexes:
      content:
        alerts:
          coldPath: $SPLUNK_DB/alerts/colddb
          homePath: $SPLUNK_DB/alerts/db
          thawedPath: $SPLUNK_DB/alerts/thaweddb
        cm_events:
          coldPath: $SPLUNK_DB/cm_events/colddb
          homePath: $SPLUNK_DB/cm_events/db
          thawedPath: $SPLUNK_DB/cm_events/thaweddb
        cm_meta:
          coldPath: $SPLUNK_DB/cm_meta/colddb
          homePath: $SPLUNK_DB/cm_meta/db
          thawedPath: $SPLUNK_DB/cm_meta/thaweddb
        cm_metrics:
          coldPath: $SPLUNK_DB/cm_metrics/colddb
          datatype: metric
          homePath: $SPLUNK_DB/cm_metrics/db
          thawedPath: $SPLUNK_DB/cm_metrics/thaweddb
        default:
          remotePath: volume:remote_store/$_index_name
          repFactor: auto
        volume:remote_store:
          path: s3://bucket_name
          remote.s3.encryption: sse-s3
          storageType: remote
      directory: /opt/splunk/etc/system/local/

I understand that yaml loader makes no guarantee around order, and you take it as you get it. Ordered dicts are going to be a feature of the language in 3.7 but that's a way away.

Nevertheless, this would be a really nice thing to be able to do. The documentation is pretty clear not all configuration can be done via default.yml so I know it's not a bug, but it would be a very helpful nice-to-have.

I've found what seems like a fairly straight-forward way to overcome this that doesn't appear to have side effects but this seemed like a rather rude PR to just request without even asking if it's ok to proceed.

[nwang92]

I don't think this fully addresses your needs, but as part of https://github.com/splunk/splunk-ansible/pull/442 we're changing/supporting splunk.conf as both a map (as it currently exists today) as well as an array. I'll modify the docs to include examples of both, but given that an array is now supported, you should be able to achieve what you're looking for at the cost of some verbosity:

splunk:
  conf:
  - key: indexes
     value:
       directory: /opt/splunk/etc/system/local/
       content:
         default:
           remotePath: "volume:remote_store/$_index_name"
           repFactor: auto
  - key: indexes
     value:
       directory: /opt/splunk/etc/system/local/
       content:
         "volume:remote_store":
           storageType: remote
           path: "s3://bucket-name"
           remote.s3.encryption: sse-s3
...

But as you mentioned, this is more-or-less part of the way YAML is structured. You should be able to keep your default.yml in an ordered fashion, but that doesn't necessarily guarantee order when the YAML is loaded + acted on by Ansible. Ultimately it shouldn't quite matter to Splunk the order in which conf file stanzas are written, but unfortunately there may not be much to do around this.