The reload configuration with the worker tag failed

[xidiandb]

Describe the bug

I used a configuration with the worker tag, which runs on startup but not on reload.

To Reproduce

Use my configuration to start and reload

Expected behavior

fluent/log.rb:371:error: Failed to reload config file: specified worker_id<0> collisions is detected on directive. Available worker id(s): []

Your Environment

- Fluentd version: 1.11.4
- TD Agent version:
- Operating system: ubuntu 18.04
- Kernel version:5.4.61-050461-generic

Your Configuration

<system>
  workers 1
  rpc_endpoint "#{ENV['POD_IP']}:24444"
</system>
<label @FLUENT_LOG>
<match fluent.*>
  @type null
</match>
</label>   
<worker 0>
<source>
  @type sample
  sample {"hello borg ooo":"world"}
  rate 1
  tag sample.ni.hao
</source>
</worker>
<worker 0>
<match sample.*.*>
  @type stdout
</match>
</worker>

Your Error Log

fluent/log.rb:371:error: Failed to reload config file: specified worker_id<0> collisions is detected on <worker> directive. Available worker id(s): []

Additional context

No response

[kenhys]

Use the following, do not define multiple <worker 0>.

<system>
  workers 1
  rpc_endpoint "#{ENV['POD_IP']}:24444"
</system>
<label @FLUENT_LOG>
  <match fluent.*>
    @type null
  </match>
</label>   
<worker 0>
  <source>
    @type sample
    sample {"hello borg ooo":"world"}
    rate 1
    tag sample.ni.hao
  </source>
  <match sample.*.*>
    @type stdout
  </match>
</worker>

[xidiandb]

@kenhys But the first time it started, there was no problem, only on reload ，Moreover, my configuration is very complex, which is divided into multiple files, and some configurations cannot be written to a worker tag. I would like to know if this boot is ok and if it is a bug when it reloads

[xidiandb]

I want to add configuration dynamically by adding files, each with worker tags. Is that not supportive? But why is there no problem at startup, only on reload

[kenhys]

Hmm, I've overlooked it.

[kenhys]

https://github.com/fluent/fluentd/blob/master/lib/fluent/supervisor.rb#L290-L303 It seems that Fluent::Engine.reload_config raise it.

[ashie]

I cannot reproduce it by HUP signal, but can reproduce by USR2 signal.

[ashie]

There are 2 places which show such message:

https://github.com/fluent/fluentd/blob/1b46fe0a6429aca531c175609a4e87459d8c1a1f/lib/fluent/static_config_analysis.rb#L81

https://github.com/fluent/fluentd/blob/1b46fe0a6429aca531c175609a4e87459d8c1a1f/lib/fluent/root_agent.rb#L95

and former one is used in this case.

[ashie]

Fluent::StaticConfigAnalysis is used only on graceful-reload. It's added for implementing graceful-reload feature.

[ashie]

<worker> elements are parsed by the following code on start up. https://github.com/fluent/fluentd/blob/8f990b8e496833e89ec109c58494f72e4885b4ac/lib/fluent/root_agent.rb#L69-L124

Fluent::StaticConfigAnalysis seems be quit different than it.

[ashie]

I want to add configuration dynamically by adding files, each with worker tags. Is that not supportive? But why is there no problem at startup, only on reload

Hmm, it's ambiguous whether multiple <worker> for same ID is supported or not. The duplication check is introduced at #2292 only for <worker n-m> syntax. <worker n> syntax is allowed multiple both before & after it.

[ashie]

I think it would be better that check duplication for <worker n> syntax too, but show only warning and don't block loading to keep compatibility.