search

influxdata / influxdb-client-python

InfluxDB 2.0 python client
https://influxdb-client.readthedocs.io/en/stable/
MIT License
724 stars 187 forks source link

Missing entrys in bool column are replaced with "None" (DataFrame) #550

Closed flyingdodo11 closed 1 year ago

flyingdodo11 commented 1 year ago

Specifications

Code sample to reproduce problem

                    |> range(start: {start_time}, stop: {end_time}) \
                    |> filter(fn: (r) => r._measurement == "{m}" ) \
                    |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value") \
                    |> group() \
                    |> sort(columns: ["_time"], desc: false) \
                    |> drop(columns: ["_start", "_stop", "_measurement"]) '

                    df = self.query_api.query_data_frame(query=query,data_frame_index=['_time'])

Expected behavior

My Measurement includes many fields and some of them are bool fields. Not every row has a value in every field. I expect that when reading and converting into a dataframe object, the missing entries of the bool columns are occupied as NaN

Actual behavior

However, the missing entries are set as None, which means that my column is no longer of the data type bool, but of the data type string, which leads to errors during further processing.

See here:

<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">

_time | Temp | Error | Control -- | -- | -- | -- 2022-05-13 09:25:21.243512+00:00 | 80.3 | None | None 2022-05-13 09:25:21.296167+00:00 | NaN | False | None 2022-05-13 09:25:21.306052+00:00  26 | NaN | None | True

Datatypes:

Temp float64 Error object Control object

Additional info

No response

bednar commented 1 year ago

Hi @flyingdodo11,

I have prepared a fixed version within #584. Development version of the client will be available after #584 acceptance, stable version in regular release cycle.

You could use a fixed version by:

pip install git+https://github.com/influxdata/influxdb-client-python.git@pandas-query-bools

Regards