bucket stays empty after sending dataframe.

[shyney7]

After sending the pandas dataframe like in the examples I only see the measurement name 'gpslive2' without any fields. It seems that no data is being send besides the name.

This is the code I'm using

import pandas as pd
from datetime import datetime, timedelta
from influxdb_client import InfluxDBClient, WriteOptions

df = pd.read_csv(r'./gpslive/gpstobfilt.csv')

start_time = datetime(2024, 4, 14, 15)
df['time'] = [(start_time + timedelta(seconds=i)).timestamp() for i in range(len(df))]
df = df.set_index('time')

client = InfluxDBClient.from_config_file(r'./gpslive/config.ini')

with client as _client:
    with _client.write_api(write_options=WriteOptions(
        batch_size=500,
        flush_interval=10_000,
        jitter_interval=2_000,
        retry_interval=5_000,
        max_retries=5,
        max_retry_delay=30_000,
        max_close_wait= 300_000,
        exponential_base=2

    )) as _write_client:
        _write_client.write("gpslivetest", "roboflex", record=df, data_frame_measurement_name='gpslive2')

This is my dataframe:

               latitude  longitude    altitude  total_counts
time
1.713100e+09  51.402865   6.903985  466.414490          0.95
1.713100e+09  51.410175   6.906332  467.008087          1.75
1.713100e+09  51.415283   6.918162  471.229858          3.60
...

InfluxDB v2.7.6

[bednar]

Hi @shyney7,

thanks for using our client.

It should be caused by wrong precision or by something else. Please try to set precision by:

_write_client.write("gpslivetest", "roboflex", record=df, data_frame_measurement_name='gpslive2', write_precision='ns')

Can you please share how looks like your ./gpslive/gpstobfilt.csv?

Also can you share the debug output of the client?

InfluxDBClient.from_config_file(r'./gpslive/config.ini', debug=True)

Regards

[shyney7]

@bednar Hi thank you for this library and all your work! And also thx for the quick reply! csv file:

latitude,longitude,altitude,total_counts
51.402865,6.903985,466.41449,0.9500000000000002
51.410175,6.906332,467.008087,1.7500000000000002
51.415283,6.918162,471.229858,3.5999999999999996
51.415491,6.919209,474.410004,2.1999999999999997
51.418973,6.93744,492.424408,2.2499999999999996
51.418973,6.93744,492.424408,3.2499999999999996
51.425427,6.98082,494.159302,5.35
51.422828,6.992541,493.392761,4.8999999999999995
51.41925,7.002204,512.323059,4.649999999999999
51.418847,7.003249,515.516052,4.899999999999999
51.418847,7.003249,515.516052,4.3999999999999995
51.418447,7.004292,518.526978,3.599999999999999
51.412791,7.018571,555.917542,5.749999999999999
51.412382,7.019565,558.280884,5.499999999999999
51.412382,7.019565,558.280884,5.75
51.403315,7.041241,595.847351,6.749999999999999
51.402888,7.042232,596.526245,6.05
...

debug output:

latitude=50.689942,longitude=7.448939,total_counts=0.3 1713205323\ngpslive4 altitude=2301.418701,latitude=50.690576,longitude=7.448668,total_counts=0.2 1713205324\ngpslive4 altitude=2301.427734,latitude=50.691211,longitude=7.448399,total_counts=0.1 1713205325\ngpslive4 altitude=2301.557129, ...
... 1713205498\ngpslive4 altitude=2301.988281,latitude=50.801363,longitude=7.402485,total_counts=0.05 1713205499'
<<< Response: 204
<<< X-Influxdb-Build: OSS
<<< X-Influxdb-Version: v2.7.6
<<< Date: Tue, 16 Apr 2024 10:02:30 GMT
<<< Body:
>>> Request: 'POST http://10.40.14.18:8086/api/v2/write?org=roboflex&bucket=gpslivetest&precision=ns'
>>> Content-Type: text/plain
>>> Accept: application/json
>>> Authorization: ***
>>> User-Agent: influxdb-client-python/1.41.0
>>> Body: b'gpslive4 altitude=2301.929443,latitude=50.801996,longitude=7.402239,t ...
... longitude=6.939595,total_counts=3.35 1713206904\ngpslive4 altitude=163.582138,latitude=51.406682,longitude=6.939595,total_counts=2.1 1713206905'
<<< Response: 204
<<< X-Influxdb-Build: OSS
<<< X-Influxdb-Version: v2.7.6
<<< Date: Tue, 16 Apr 2024 10:02:31 GMT
<<< Body:

Ive shorten the output to the relevant parts with... setting write_precicion='ns' seems not to help

[bednar]

Sorry for the late. You have to use second precision because you increment your time by secs: [(start_time + timedelta(seconds=i)).timestamp() for i in range(len(df))]. Try this:

_write_client.write("gpslivetest", "roboflex", record=df, data_frame_measurement_name='gpslive2', write_precision =WritePrecision.S)

[shyney7]

@bednar thank you! this solved the issue!