influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.66k stars 3.54k forks source link

Changing the Aggregation Selection in a Check Can Cause the Check to Fail #20605

Open rickspencer3 opened 3 years ago

rickspencer3 commented 3 years ago
  1. Have a measurement where the field value is an integer
  2. Create a Check, and leave the default aggregation to mean()
  3. Let the check run once
  4. Reopen the check, and change the aggregation to max()

Result: The check fails because Max() now returns an integer, and the Check task tries to write an integer to a field that was first written with a float. This information is available by searching the _task bucket.

Expected: The Check does not fail. At a minimum, provide an error message so the user has a hope of figuring out what the problem is.

From the task log:

"message":"could not execute task run; Err: partial write: field type conflict: input field \"n\" on measurement \"statuses\" is type float, already exists as type integer dropped=1: partial write: field type conflict: input field \"n\" on measurement \"statuses\" is type float, already exists as type integer dropped=1"}]
shades2 commented 3 years ago

We've encountered the issue ourselves. Seemed to result in very high SNMP process CPU usage. Changing SNMP monitored router configs / aggregation may trigger things also, which makes it tricky.

bartoszpijet commented 2 years ago

For me removal of /var/lib/influxdb/engine fixes the issue (of cause all collected data is being lost in that process, didn't care that much)