[Threat Intelligences Utilities]: Intelligence Dashboard in Elastic Security Broken

[Danouchka]

Integration Name

Threat Intelligence Utilities [ti_util]

Integration Version

1.6.0

Agent Version

8.14.1

Agent Output Type

elasticsearch

Elasticsearch Version

8.14.3

OS Version and Architecture

Elastic Cloud Hosted GCP Belgium

Software/API Version

No response

Error Message

Event Original

No response

What did you do?

Nothing special Upgraded to the latest integrations

What did you see?

A conflicting type is at the origin of the issue

What did you expect to see?

Actually if we take the logs-ti_abusech.threatfox template, it is made of the following component templates

• logs@settings
• logs-ti_abusech.threatfox@package
• logs-ti_abusech.threatfox@custom
• ecs@mappings
• .fleet_globals-1
• .fleet_agent_id_verification-1

In none of them , first_seen or last_seen is explicitely defined as a date

Please is it possible to set explicitely in the mappings of all threat intelligence templates or in ecs@mappings that the fields threat.indicator.first_seen and threat.indicator.last_seen are typed as date ?

Thank you

Anything else?

No response

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[Danouchka]

cc @jamiehynds @dhru42 @PhilippeOberti

[jamiehynds]

Thanks for reporting @Danouchka

@kcreddy could this be a result of the transforms added to the TI packages, and the dashboard needs adjusting as a result?

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[kcreddy]

The threat ECS date fields such as last_seen, first_seen, modified_at are not mapped into ecs@mappings component template as they don't seem to match any patterns defined in here.

While this gets fixed inside the ecs@mappings, I created a temporary fix for the AbuseCH integration: https://github.com/elastic/integrations/pull/10637

[kcreddy]

The fix is also needed for rest of the ti_* packages.

[kcreddy]

@Danouchka, The PR for Abusech: https://github.com/elastic/integrations/pull/10637 is merged and the mapping fix available in 2.3.1 Also MISP is also fixed in https://github.com/elastic/integrations/pull/10638 and available in version 1.35.1.

[kcreddy]

Closing this issue as fixed by: https://github.com/elastic/integrations/pull/10637 (for Abusech), https://github.com/elastic/integrations/pull/10638 (for MISP) https://github.com/elastic/integrations/pull/10674 (for all other TI providers)