search

elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.2k stars 3.5k forks source link

Repeated deprecation setting error on main after #15679 #16505

Open andsel opened 11 hours ago

andsel commented 11 hours ago

Description of the problem including expected versus actual behavior: After merge of PR #15679 which introduced new Java classes for base, aliased and Boolean setting translating from Ruby existing code, if Logstash runs with the reload of a pipeline flag (both as command line --config.reload.automatic or configuring it in config/logstash.yml, it starts repeating the following error log:

[2024-10-03T14:03:12,018][WARN ][org.logstash.settings.DeprecatedAlias] The value of setting `api.enabled` has been queried by its deprecated alias `http.enabled`. Code should be updated to query `api.enabled` instead

This happens only when the pipeline is configured in config/pipelines.yml, both as config.string or with path path.config. Instead, it doesn't happen when running directly the pipeline form the file with logstash -f simple_pipeline.conf

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including (e.g.) pipeline definition(s), settings, locale, etc. The easier you make for us to reproduce it, the more likely that somebody will take the time to look at it.

  1. edit config/pipelines.yml adding 
    - pipeline.id: test
config.string: "input{ tcp { port => 12345 } } output { stdout{} }"
  2. Run Logstash with 
    bin/logstash --config.reload.automatic

If the reload it's not enabled, it doesn't happen. Also note that it happend every 3 seconds (the default value of config.reload.interval) and appears always twice

Provide logs (if relevant):

[2024-10-03T14:12:36,209][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:test], :non_running_pipelines=>[]}

[2024-10-03T14:12:39,242][WARN ][org.logstash.settings.DeprecatedAlias] The value of setting `api.enabled` has been queried by its deprecated alias `http.enabled`. Code should be updated to query `api.enabled` instead
[2024-10-03T14:12:39,245][WARN ][org.logstash.settings.DeprecatedAlias] The value of setting `api.enabled` has been queried by its deprecated alias `http.enabled`. Code should be updated to query `api.enabled` instead

[2024-10-03T14:12:42,234][WARN ][org.logstash.settings.DeprecatedAlias] The value of setting `api.enabled` has been queried by its deprecated alias `http.enabled`. Code should be updated to query `api.enabled` instead
[2024-10-03T14:12:42,235][WARN ][org.logstash.settings.DeprecatedAlias] The value of setting `api.enabled` has been queried by its deprecated alias `http.enabled`. Code should be updated to query `api.enabled` instead

[2024-10-03T14:12:45,230][WARN ][org.logstash.settings.DeprecatedAlias] The value of setting `api.enabled` has been queried by its deprecated alias `http.enabled`. Code should be updated to query `api.enabled` instead
[2024-10-03T14:12:45,233][WARN ][org.logstash.settings.DeprecatedAlias] The value of setting `api.enabled` has been queried by its deprecated alias `http.enabled`. Code should be updated to query `api.enabled` instead
andsel commented 8 hours ago

This problem raises from the fact the Agent cyclically (if autoreload is enabled) reload and resolve pipeline configuration, in checking if Settings instances has changed, it invokes the equals method which goes deep to the DeprecatedAlias.value and so the log.

In more details: https://github.com/elastic/logstash/blob/e84fb458ce2f092e065c63df649222f8cbda8c44/logstash-core/lib/logstash/agent.rb#L132

https://github.com/elastic/logstash/blob/e84fb458ce2f092e065c63df649222f8cbda8c44/logstash-core/lib/logstash/agent.rb#L192

https://github.com/elastic/logstash/blob/e84fb458ce2f092e065c63df649222f8cbda8c44/logstash-core/lib/logstash/agent.rb#L339

https://github.com/elastic/logstash/blob/e84fb458ce2f092e065c63df649222f8cbda8c44/logstash-core/lib/logstash/agent.rb#L421

https://github.com/elastic/logstash/blob/e84fb458ce2f092e065c63df649222f8cbda8c44/logstash-core/lib/logstash/state_resolver.rb#L38

https://github.com/elastic/logstash/blob/e84fb458ce2f092e065c63df649222f8cbda8c44/logstash-core/src/main/java/org/logstash/config/ir/PipelineConfig.java#L158

https://github.com/elastic/logstash/blob/e84fb458ce2f092e065c63df649222f8cbda8c44/logstash-core/lib/logstash/settings.rb#L226-L229

In checking equality of hash map both keys and values of hash maps are accessed. The map (due to the with_deprecated_alias) contains 2 settings one for api.enabled which is the SettingWithDeprecatedAlias proxy and another for http.enabled which is the instance of DeprecatedAlias. So it seems that every time this comparison process accesses an instance of DeprecatedAlias then the value() method is also invoked.

andsel commented 7 hours ago

The motivation of access the value is that Settings#to_hash method does it:

https://github.com/elastic/logstash/blob/e84fb458ce2f092e065c63df649222f8cbda8c44/logstash-core/lib/logstash/settings.rb#L154-L161