search

influxdata / flux

Flux is a lightweight scripting language for querying databases (like InfluxDB) and working with data. It's part of InfluxDB 1.7 and 2.0, but can be run independently of those.
https://influxdata.com
MIT License
767 stars 153 forks source link

feature request: support elasticsearch search json API query #539

Open jordo1138 opened 5 years ago

jordo1138 commented 5 years ago

would be fantastic to have a native flux option to query the http api

nathanielc commented 5 years ago

@jordo1138 Can you give a fake Flux example of what you are thinking?

jordo1138 commented 5 years ago

@nathanielc sure, so from what I gather from the blogs and influxdays talks, influxdb will let you have running tasks that are "kapacitor" like "tasks" and it would be interesting for me to use flux to write something like below, not exactly sure if its the right way but taking the simple style of the sample flux query to using @cpu.usage_user like lucene does for indexed fields/ tags etc in elasticsearch directly and then we could take action on the results like in kapacitor, so it would be querying just for the purpose of a running task, not to store necessarily from ES to influx or anything, just centralizes the running task part so that we could then do notifications etc and other triggered output that eventually flux will have Definitely will spend more time on flux as I haven't had a chance to yet, but will in the new year.

from(db:"telegraf") |> filter(fn:(r) => r._measurement == "cpu" and r._field == "usage_user" and r.cpu == "cpu3") |> range(start:-1h)

TO: from(db:"elasticsearch") . //define this somewhere like es client for golang/python does |> filter(fn:(r) => r._measurement == "@_cpu" and r._field == "@cpu.usage_user" and r.cpu == "@_cpu3") |> range(start:-1h)

nathanielc commented 5 years ago

@jordo1138 Thanks for the details. This helps me understand what you are after.

A elasticsearch source makes lots of sense for Flux's goals. We are working to making writing and more importantly testing new sources easier. Once we have some of those things in place I can add some more details to this issue for what it would take to be able to query elasticsearch data from Flux. But the basic idea is that a new input needs to be added to functions/inputs that can make an elastic search query using client libs.

jordo1138 commented 5 years ago

@nathanielc cool I hadn't read enough on flux yet to understand how client libs were going to be handled but I'll dive in deeper on functions/inputs Thanks!

github-actions[bot] commented 1 day ago

This issue has had no recent activity and will be closed soon.