search

influxdata / influxdb-relay

Service to replicate InfluxDB data for high availability
MIT License
855 stars 350 forks source link

Wrong rp in influx write causing ingest issues #71

Open monitoringit opened 5 years ago

monitoringit commented 5 years ago

A client was writing data to backend influxdbs through relay using wrong retention policy (rp). Both backend Influxdbs returned 5xx errors for it. Relay then kept accruing these data points in its buffer eventually filling the buffers and all other valid writes from other clients started failing.

This implies any bad player out there can impact filling up relay buffers...is there a way for relay to forget these errors and not keep them in memory. A human error on client side config can impact the whole relay by eventually choking up its memory.

It may be by design but want to confirm ....

rockyluke commented 5 years ago

Hello @monitoringit you can find the reason here : https://github.com/vente-privee/influxdb-relay/blob/master/relay/retry.go#L65.

We're working on a fork, feel free to make issue / MR to help us improve the daemon.

lucas-santoni commented 5 years ago

Discussion on the fork : https://github.com/vente-privee/influxdb-relay/issues/12