DataDog / ansible-datadog

Ansible role for Datadog Agent
Apache License 2.0
297 stars 222 forks source link

[IMPROVEMENT] Do not sort yaml keys of check configurations #557

Closed WillBrock closed 3 weeks ago

WillBrock commented 5 months ago

to_nice_yaml will sort the keys by default which can cause the formatting of the generated yaml file to be incorrect.

In this example case when you add the log_processing_rules section it will change the order and the type: file will end up being inside the log_processing_rules section causing the checks to fail. If you remove the log_processing_rules section then the yaml file will be correct.

This is defined in the playbook:

nginx:
  init_config:
  instances:
    - nginx_status_url: "http://localhost/nginx_status"
  logs:
    - type: file
      path: /var/log/nginx/access.log
      service: nginx
      source: access
      sourcecategory: http_web_access
    - type: file
      path: /var/log/nginx/critical.log
      service: nginx
      source: critical
      sourcecategory: http_web_access
    - type: file
      path: /var/log/nginx/error.log
      service: nginx
      source: error
      sourcecategory: http_web_access
      log_processing_rules:
        - name: exclude_ckeditor
          type: exclude_at_match
          pattern: foobar

Here is the example /etc/datadog-agent/conf.d/nginx.d/conf.yaml file that gets generated when including the log_processing_rules section.

init_config: null
instances:
- nginx_status_url: http://localhost/nginx_status
logs:
- path: /var/log/nginx/access.log
  service: nginx
  source: access
  sourcecategory: http_web_access
  type: file
- path: /var/log/nginx/critical.log
  service: nginx
  source: critical
  sourcecategory: http_web_access
  type: file
- log_processing_rules:
  - name: exclude_ckeditor
    pattern: foobar
    type: exclude_at_match
  path: /var/log/nginx/error.log
  service: nginx
  source: error
  sourcecategory: http_web_access
  type: file
alopezz commented 1 month ago

👋 Hi! If I understand correctly you're saying that you're second example is incorrect. How so? I've checked quickly with a yaml parser and it seems to produce the expected data structure.

WillBrock commented 1 month ago

Hi, it's been so long that I don't really remember and I don't use Datadog anymore to check on it. I should have posted the before and after output of the yaml. In general it would be better to not sort the keys anyways.

alopezz commented 1 month ago

Thanks for your response. I'll close this for now, but if anyone else requests this we can reopen and reconsider.

AliDatadog commented 3 weeks ago

Thank you for your contribution.

I could QA the change manually and validate that the CI passes properly after a rebase on main.