influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.66k stars 3.54k forks source link

InfluxDB 2.x: Query measurements with different data types = hangs/timeout #20980

Open R-Studio opened 3 years ago

R-Studio commented 3 years ago

Unfortunately I have a measurement that have fields with an number as the “value” but the type is a string instead of an integer. (This because I made a mistake in the script that writes the data to the InfluxDB.)_

What I want: I want to change the data type of this fields for the whole bucket. My Idea was to get all the data, convert the data types and writes it to another bucket like this:

    datatoconvert = from(bucket: "telegraf")
      |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
      |> filter(fn: (r) => r._measurement == "asset_vm_cmdb")
      |> toInt()

    datatoconvert
      |> filter(fn: (r) => exists r._value)
      |> to(bucket: "typeconversion", org: "<MY_ORG>")

Steps to reproduce:

  1. Write data to an InfluxDB: Some fields _value as integer (double in Influx) and some fields _value a string.
  2. Query this data in the InfluxDB Explore GUI and write this to a new bucket

Expected behavior: I can query the data, convert the data types of the all "_values" and write this to a new bucket.

Actual behavior: The InfluxDB Explore GUI hangs after some seconds and did not respond. Even when I use a short timerange this always hangs. More information's in the community post (https://community.influxdata.com/t/change-type-from-string-to-integer)

Environment info:

Logs: I found no error logs with sudo journalctl -u influxdb.service.

First, please read this community post: https://community.influxdata.com/t/change-type-from-string-to-integer

R-Studio commented 3 years ago

@danxmoran any news on this? This is really urgent for us because our Downsampling InfluxDB Tasks are failing because of this different data types.

R-Studio commented 3 years ago

I try to find a workaround by filtering only data with wrong data types (data type of "_value" = string, but contains only digits). For this filtering I tried to use the function strings.isDigit() to filter this, but this function only works for a single character. Is there a way to check multiple character with this function? Or is there a way to filter data based data type? (something like filter(fn: (r) => r._value == datatype(string))

R-Studio commented 3 years ago

@danxmoran any news on this?

R-Studio commented 2 years ago

??