reloader should also check the final fluent.conf file for errors

[alex-vmw]

reloader should also check the final fluent.conf file for errors.

We had an issue where a user set the same @id in multiple fluendConfigs of different namespaces. While individual fluendConfigs had no errors, the final fluent.conf file had errors due to duplicated plugin id:

2021-09-22 00:12:02 +0000 [error]: config error file="/fluentd/etc/fluent.conf" error_class=Fluent::ConfigError error="Duplicated plugin id `out_vmw_li_cmbu-symphony-logs`. Check whole configuration and fix it."

P.S. If the above can't be implemented (maybe it is too hard to determine which configs broke the final fluent.conf file), I think at least kfo should disallow @id field and filter it out if users decide to set it.

[josephmcasey]

I recently ran into this problem due to the following Prometheus rule having an identical @id which was added between 1.16.7 and 1.16.8 . As an intermediate solution for this problem, it would be great if the README.md included a full list of every @id added by this repository as a more immediate checklist for consumers of the artifact.

<source>
  @type prometheus_tail_monitor
</source>

<filter **>
  @type prometheus
  @id filter_prometheus
  @log_level warn

  <metric>
    name fluentd_input_status_num_records_total
    type counter
    desc The total number of incoming records
    <labels>
      tag ${tag_parts[0]}
      hostname ${hostname}
    </labels>
  </metric>
</filter>