search

influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.18k stars 5.52k forks source link

Elasticsearch output cannot be used with data streams #15051

Closed cnemo-cenic closed 3 months ago

cnemo-cenic commented 3 months ago

Use Case

Setup an index template and data stream in Elasticsearch. Configure Telegraf to index into Elasticsearch with the "create" optype.

Expected behavior

Data ingests into Elasticsearch correctly.

Actual behavior

Elasticsearch returns an error, because only the "create" operation type can be used with data streams (not the default "index" operation type).

Additional info

https://github.com/olivere/elastic/issues/1464

Indexing requests add documents to a data stream. These requests must use an op_type of create.

https://www.elastic.co/guide/en/elasticsearch/reference/7.17/set-up-a-data-stream.html

powersj commented 3 months ago

Need the CLA signed, then we can wrap up reviews of the PR. Thanks!

telegraf-tiger[bot] commented 3 months ago

Hello! I am closing this issue due to inactivity. I hope you were able to resolve your problem, if not please try posting this question in our Community Slack or Community Forums or provide additional details in this issue and reqeust that it be re-opened. Thank you!

powersj commented 3 months ago

If you ever want to sign the CLA, we would be happy to revisit this!

cnemo-cenic commented 3 months ago

@powersj Thanks. I definitely want that to happen, and I am optimistic it will be done at some point. But as of yet my employer has not been able to execute the decision.

MartVisser commented 1 month ago

Any update on this? I would like to use this feature.