threat.indicator.first_seen ECS Field Coming as a Keyword on Kibana

[mohitjha-elastic]

threat.indicator.first_seen is coming as a keyword on kibana. Its type is mentioned as a date in the ECS documentation- https://www.elastic.co/guide/en/ecs/current/ecs-threat.html#field-threat-indicator-first-seen

Tried the following ways:

1. Parsing the threat.indicator.first_seen field using the date processor.
2. Using set processor to set the parsed date field to the threat.indicator.first seen

Using both the above ways, we are getting threat.indicator.first_seen as a keyword on kibana.

We have also observed the same behavior in threat.indicator.last_seen and threat.indicator.modified_at. They are also coming as keyword on the kibana.

Please find the attached screenshots for the same:

[andrewkroh]

This isn't a problem with ECS AFAICT. I think the problem is that your index template for that data stream is missing a mapping for the those fields and they are defaulting to keyword.

[brett-fitz]

As @andrewkroh noted this isn't a problem with the ECS artifact but to double check that your index template includes the required threat field mappings and your index template is being applied to the backing datastream/indice (hint: check priority of your index template). If this is not the case then elasticsearch will default to dynamic mapping.

However, @mohitjha-elastic looks like you are using the Crowdstrike threat intel integration from elastic, this doesn't seem to include the threat fields for falcon or fdr? This could likely be the issue and an issue ticket for that repo may be more applicable.

[mohitjha-elastic]

@andrewkroh @brett-fitz We have been facing this issue while developing the ti_crowdstrike integration for which we have already raised the PR in the integration repo. We are using the import_mapping: true to dynamically map all the ECS fields according to their types.

As you can see here, we have used the date processor for parsing the created_on field and then used the set processor so it must set the threat.indicator.first_seen with the type date but it's coming as a keyword on kibana (Attached image for the same in the issue description).

Please let us know if you need anything more from our side.

[brett-fitz]

@mohitjha-elastic Unfortunately I haven't had any experience with the dynamic ECS mapping process but @P1llus would likely be a great resource here.

REF: https://github.com/elastic/integrations/issues/5055

[andrewkroh]

I'm transferring this issue from elastic/ecs to elastic/elastic-package because I believe the problem is that when import_mappings is used that the included dynamic mappings may not include a mapping rule for {first,last}_seen and modified_at.

Relates to https://github.com/elastic/elastic-package/blob/85d6fcacad736e543e459a044a5e0fa48b5d43c6/internal/builder/_static/ecs_mappings.yaml.