search

elastic / elasticsearch

Free and Open, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.43k stars 24.57k forks source link

Expose Grok as a painless function to the ingest node context #78008

Open jakelandis opened 2 years ago

jakelandis commented 2 years ago

The ability to grok a log message inside an "if" condition could help to make better decisions about which index/data stream to route a document without addinging additional information to the source document. This is likely most useful in combination with Runtime fields and source only indexes.

Runtime fields already supports grok via painless, so this is in large part an effort to expose grok via the ingest node context for use in an "if" or script processor.

related: #73468, #63798

elasticmachine commented 2 years ago

Pinging @elastic/es-data-management (Team:Data Management)

elasticmachine commented 2 years ago

Pinging @elastic/es-core-infra (Team:Core/Infra)

felixbarny commented 2 years ago

@jakelandis runtime fields supporting grok in scripts now: https://www.elastic.co/guide/en/elasticsearch/painless/master/painless-runtime-fields.html

Is that something we can easily leverage in the context of processor conditions?