[DISCUSS] consider exposing deprecation warnings via API

[yaauie]

In https://github.com/elastic/logstash/pull/11260 (7.6, 8.x), we introduced a deprecation logger, and made it available to plugins with the Deprecation Logger Support mixin.

This allows both Logstash core and plugins to emit separately-routed log events when they encounter a configuration that is scheduled for breaking changes.

Most deprecation logs occur at either process or pipeline start-up, and can easily get lost when the Logstash process is left running for long periods of time due to log retention policies. Should we intercept deprecation logs and expose them in an API, so that the Kibana Upgrade Assistant can be useful in Stack upgrades that include Logstash?

If so, what is the best plan of attack:

• intercept before the log has been written by modifying the DeprecationLogger class
  • con: we don't have direct access to logging context like thread id, plugin id, etc.
  • pro: in-place upgrades get this implementation "for free"
• intercept by implementing an AbstractAppender to be registered in log4j config
  • con: in-place upgrades won't consume config changes
  • pro: full logging context available

[andsel]

In the first case we can retrieve the plugin.id using

org.apache.logging.log4j.ThreadContext.get("plugin.id");

in the DefaultDeprecationLogger

In the second case we can analyze the log4j2 config on startup in the same way we plan to do in https://github.com/elastic/logstash/pull/12512/files#r543265119, informing the user to update the appender definition in their log4j2.properties