How to avoid duplicating Formatter Configuration

[yordis]

Hey team! Currently, I have the following config files (test and prod).

# config/dev.exs

config :logger, :default_formatter,
  truncate: :infinity,
  format: "$time $metadata[$level] $message\n",
  metadata: [
    :request_id,
    :trace_id,
    :span_id
    # ... even more keys
  ]

And

# config/prod.exs

config :logger, :default_handler,
  formatter:
    {LoggerJSON.Formatters.Basic,
     metadata: [
       :request_id,
       :trace_id,
       :span_id
       # ... even more keys
     ]}

Where I would duplicate the configuration in both config files, ideally, I wish to use config/config.exs only or be able to configure the :default_formatter metadata only to avoid forgetting to change all the required files to keep it in sync.

What would you suggest here to avoid duplication?

[AndrewDryga]

Hello 👋, the reason why it wasn't the issue for us is that for JSON we keep metadata: {:all_except, [:conn, :socket]} because we want to keep as much data as we can, while dev/test logs (from config.exs) are typically very minimal to keep readability.

I think what we can do is to either try to read config from [:logger, :default_formatter, :metadata] by default (but I think that might cause issues) or introduce a new syntax for metadata that would looks something like {:from_application_env, [:logger, :default_formatter, :metadata]}. WDYT?

[yordis]

{:from_application_env, [:logger, :default_formatter, :metadata]}

I think that would work; I was thinking something around those lines, maybe metadata: :default_formatter?

I am unsure of the technical details, so I trust your judgment!

[anthonator]

I think it would be great if the metadata from the default formatter could somehow be inherited by LoggerJSON.

I currently do the following to avoid duplicating logger metadata config.

logger_metadata = [:request_id]

if Mix.env() == :prod do
  config :logger, :default_handler,
    formatter: {LoggerJSON.Formatters.Basic, [metadata: logger_metadata]}
else
  config :logger, :default_formatter,
    format: "$time $metadata[$level] $message\n",
    metadata: logger_metadata
end

[AndrewDryga]

Pushed this feature to the main, please give it a try. The obvious downside is that we will have to read from application env on every logged message (which is an :ets table) since there is no way to initialize and keep the state for the :logger_formatter behavior.

[AndrewDryga]

The feature will be released in 6.2.0.