target check confusing & printed multiple times (on reload)

[kares]

... taken from https://github.com/logstash-plugins/logstash-codec-json/pull/37#issuecomment-870451875

The mixin (in 1.3.0) shipped a target check helper, which hooks up plugin.register. However, on plugin reload the missing target warning lacks context information (such as the plugin id) :

[INFO ][logstash.codecs.json ] ECS compatibility is enabled but target option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the target option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)

[wallrik]

When working on this, could we perhaps also get a value for target to allow a conscious decision to place data in the top-level of the event? I'm importing ECS-compliant files and would rather not see the message at all. 😕 It does say to ignore it, but my logs are completely full with just this message.

Thanks 🙏😇

[ebenezar-mccoy]

In my case

I had to enable ecs_compatibility as it was required after enabling data_stream in logstash Then I started getting the errors like everyone else.

setting additional_codecs => {} solved the issue for me. Default value is {"application/json"=>"json"}

input{
    http {
      additional_codecs => {}
    }

[cawoodm]

This is driving me crazy - it's logging this missage literally MILLIONS of times - about 100 times a second.

  http {
    port => 8070
    codec => json_lines {}
    ecs_compatibility => "disabled"
  }

When I remove or change ecs_compatibility to any other value my pipeline fails. With disabled however my logs fill up with this garbage. Logstash 7.17.1

Any help would be appreciated.

[lilykozharova]

Has anything been done about this? Is there a way to disable that log? It is still happening in logstash 8.6.2.

[mashhurs]

Has anything been done about this? Is there a way to disable that log? It is still happening in logstash 8.6.2.

I failed to reproduce the case with both 7.17.1 and 8.6.2. Can you please share your logstash.yml & http-input configs along with the debug logs?

[kkornienko-aparavi]

Thanks to @ebenezar-mccoy ! Adding additional_codecs => {} helped getting rid of repeating warnings.

My settings: logstash.yml - nothging regarding ecs

pipeline:

- pipeline.id: 'main'
  path.config: '/usr/share/logstash/pipeline/main'
  pipeline.ecs_compatibility: 'v1'

input:

input {
  http {
    port => 443
    ecs_compatibility => disabled

    codec => json {
      ecs_compatibility => disabled
      target => ""
    }
  }
}

this combination of settings was producing a warning on each incoming document