invalid composite mappings for [logs-fortinet.fortigate.event]

[evik-santika-ioh]

When run

PUT _index_template/logs-fortinet.fortigate.event
{
  "priority": 200,
  "index_patterns": [
    "logs-fortinet.fortigate.event*"
  ],
  "data_stream": {
    "hidden": false,
    "allow_custom_routing": false
  },
  "composed_of": [
    "ecs-base",
    "ecs-user",
    "ecs-user_agent",
    "ecs-observer",
    "ecs-destination",
    "ecs-source",
    "ecs-network",
    "ecs-error",
    "ecs-url",
    "ecs-rule",
    "ecs-data_stream",
    "ecs-organization",
    "ecs-ecs",
    "ecs-host",
    "logs-fortinet.fortigate.event@ilm",
    "strings_as_keyword@mappings",
    "auto_expand_replicas@settings",
    "refresh_interval@settings",
    "logs-fortinet.fortigate@mappings",
    "ecs-log-modified",
    "ecs-event-modified",
    "ecs-file-modified",
    "synthetic_source@mappings"
  ]
}

i got error

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "composable template [logs-fortinet.fortigate.event] template after composition with component templates [ecs-base, ecs-user, ecs-user_agent, ecs-observer, ecs-destination, ecs-source, ecs-network, ecs-error, ecs-url, ecs-rule, ecs-data_stream, ecs-organization, ecs-ecs, ecs-host, logs-fortinet.fortigate.event@ilm, strings_as_keyword@mappings, auto_expand_replicas@settings, refresh_interval@settings, logs-fortinet.fortigate@mappings, ecs-log-modified, ecs-event-modified, ecs-file-modified, synthetic_source@mappings] is invalid"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "composable template [logs-fortinet.fortigate.event] template after composition with component templates [ecs-base, ecs-user, ecs-user_agent, ecs-observer, ecs-destination, ecs-source, ecs-network, ecs-error, ecs-url, ecs-rule, ecs-data_stream, ecs-organization, ecs-ecs, ecs-host, logs-fortinet.fortigate.event@ilm, strings_as_keyword@mappings, auto_expand_replicas@settings, refresh_interval@settings, logs-fortinet.fortigate@mappings, ecs-log-modified, ecs-event-modified, ecs-file-modified, synthetic_source@mappings] is invalid",
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "invalid composite mappings for [logs-fortinet.fortigate.event]",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "field [error.stack_trace] of type [wildcard] doesn't support synthetic source"
      }
    }
  },
  "status": 400
}

after i remove

"synthetic_source@mappings"

this success.

my Question

Why component templates "synthetic_source@mappings" give above error. i check, i'm already add this component templates.

Thank you

[enotspe]

Just delete error.stack_trace field from ecs-error component template and run the command again. We dont use that field anyways.

synthetic_source helps saving 50% on disk ussage. So it is worth it. The problem is that it just supports some field types

[evik-santika-ioh]

now we get this error after fix ecs.error

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "composable template [logs-fortinet.fortigate.event] template after composition with component templates [ecs-base, ecs-user, ecs-user_agent, ecs-observer, ecs-destination, ecs-source, ecs-network, ecs-error, ecs-url, ecs-rule, ecs-data_stream, ecs-organization, ecs-ecs, ecs-host, logs-fortinet.fortigate.event@ilm, strings_as_keyword@mappings, auto_expand_replicas@settings, refresh_interval@settings, logs-fortinet.fortigate@mappings, ecs-log-modified, ecs-event-modified, ecs-file-modified, synthetic_source@mappings] is invalid"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "composable template [logs-fortinet.fortigate.event] template after composition with component templates [ecs-base, ecs-user, ecs-user_agent, ecs-observer, ecs-destination, ecs-source, ecs-network, ecs-error, ecs-url, ecs-rule, ecs-data_stream, ecs-organization, ecs-ecs, ecs-host, logs-fortinet.fortigate.event@ilm, strings_as_keyword@mappings, auto_expand_replicas@settings, refresh_interval@settings, logs-fortinet.fortigate@mappings, ecs-log-modified, ecs-event-modified, ecs-file-modified, synthetic_source@mappings] is invalid",
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "invalid composite mappings for [logs-fortinet.fortigate.event]",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "field [fgt.age] of type [long] doesn't support synthetic source because it ignores malformed numbers"
      }
    }
  },
  "status": 400
}

i not see fgt index in database elasticsearch.

[enotspe]

what version of ELK are you using???

[enotspe]

I got it with the "ignore malformed data" flag enabled and have no problem. ELK 8.10.4

[evik-santika-ioh]

what version of ELK are you using???

Thank you for your reply.

i'm using elasticsearch version 8.5.1

so any solution if using this version?

[enotspe]

mmm don't really know. I can not test it on that version.

my only suggestion would be not to use synthetic source component template, but that would be a huge increase on disk usage.

[enotspe]

try running the script for loading all components from scratch.