search

influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.63k stars 5.58k forks source link

Add Input plugin support to take inputs from Influxdb #12230

Closed prajyod closed 1 year ago

prajyod commented 1 year ago

Use Case

I have a scenario where I will run Tasks in InfluxDB . This tasks creates a bucket and data inside it. My end goal is to read the data created in this bucket and output to open telemetry format. As of now we don't have an option in input plugins where I can read data from InfluxDB .

Expected behavior

We should have a Input Plugin which take bucket details, authorization etc from Influxdb.

Actual behavior

NA

Additional info

No response

Hipska commented 1 year ago

Yes, it is possible but needs configuration on InfluxDB. The Edge Data Replication function in combination with telegrafs influxdb_v2_listener input.

telegraf-tiger[bot] commented 1 year ago

Hello! I recommend posting this question in our Community Slack or Community Page, we have a lot of talented community members there who could help answer your question more quickly. You can also learn more about Telegraf by enrolling at InfluxDB University for free!

Heads up, this issue will be automatically closed after 7 days of inactivity. Thank you!

prajyod commented 1 year ago

@Hipska Should it be code level change? or I can do some configurations to bring this feature up?

Hipska commented 1 year ago

Configuration. See https://www.influxdata.com/products/influxdb-edge-data-replication/ and more documentation will come with #12231

prajyod commented 1 year ago

@Hipska to be clear I want an input plugin to take data from a bucket in influxdb where I have some counter values which are already in Cloud . I need an input plugin where I can given the influxdb details and these details are fetched from this cloud instance .

Hipska commented 1 year ago

It is the other way round. You setup the influxdb_listener input plugin and setup your InfluxDB EDR to send to this telegraf instance.

srebhan commented 1 year ago

@Hipska I think @prajyod wants to take the data from the cloud and put it elsewhere, so EDR is not an option here.

prajyod commented 1 year ago

@Hipska: what @srebhan said is right. I want to take the data from the cloud and put it elsewhere.

Hipska commented 1 year ago

Oh, I didn't know InfluxDB cloud didn't have EDR feature.

prajyod commented 1 year ago

@Hipska @srebhan Will this feature be assigned to anyone ?

prajyod commented 1 year ago

@Hipska / @srebhan any update?

Hipska commented 1 year ago

Everybody is free to contribute to telegraf, we are happy to review your PR to implement this..

srebhan commented 1 year ago

@prajyod we are assigning the issues as soon as we are working on them. As you can see we have some of those feature-requests, so you can speed-up things by submitting a PR (or find someone who does) or help with other issues to free some time for us. Hammering with "any update" requests does certainly not reduce our workload. ;-)

prajyod commented 1 year ago

@srebhan : Sure I will try to work on this.

powersj commented 1 year ago

InfluxDB v3 can be queried directly using the SQL input plugin.

For InfluxDB v1 and v2, we have no plan to implement additional input plugins at this time. These versions have existed for quite some time and there has been little to no demand for an input plugin from these versions.

My suggestion is to use the client libraries to pull data down from them as you need to. The client libraries enable a user to use their favorite programming language to pull down from a specific version of InfluxDB. Additionally, they come with support for InfluxQL and Flux, allowing the user to transform and manipulate their data as needed.

If we suddenly getting sufficient demand or someone puts up a PR we would be happy to review, but otherwise closing this.