influxdata / influxdb-python

Python client for InfluxDB
MIT License
1.69k stars 520 forks source link

write_points of Pandas DataFrameClient ( Pandas 0.24.2) is not working #738

Open stevejobsmyguru opened 5 years ago

stevejobsmyguru commented 5 years ago

I tried to push Pandas Data Frame using write_points of DataFrameClient object (Python Package: 5.2.2).

It did not write and did not throw any exception in Influx log too when I did tail -f command.

Then I downgraded Pandas Package to 0.23.4. Then write_points method is able to push the data into InfluxDB. I feel that it is short term fix I figured out. But Influx client package must be fixed.

Can you pls. support on this front.

Andytmlr80 commented 5 years ago

I'm experiencing this issue as well. Our cloud platform is running Pandas 0.25.0 and the even the Pandas tutorial example from the Influxdb site does not work (https://influxdb-python.readthedocs.io/en/latest/examples.html#tutorials-pandas).

Reverting to Pandas 0.23.4 does work, however that's not a practical solution for us.

s-okmt commented 5 years ago

I had the same situation. I reviewed codes & found it's caused by a change on DataFrame.to_dict(). You may specify protocol="json" for write_points. If so, a tentative solution is to use protocol="line", since a problem is in a method for conversion from dataframe to json.

Before pandas v0.24.0 to_dict for empty dataframe returns an array of empty dicts & its quantity is same as the number of row of the dataframe. From v0.24.0, it returns an empty array, without any dict. In _convert_dataframe_to_json() (a method called from write_points), to_dict is used to generate some variables & it is zipped with other variables. Due to the change on pandas, the zip doesn't work well now, i.e. returns an empty iterator.

As the line protocol option on write_points does not use to_dict, I could confirm my code worked well with it (my influxdb package is v5.2.2 & pandas is v0.24.2). I would like to tentatively use this instead of json option.

sebito91 commented 5 years ago

Yes, this needs to be fixed. I'm looking into this now...thanks a lot for your patience!

stevejobsmyguru commented 5 years ago

Thx for reply. We will try the 'line' protocal in write_points as the work around. We will be looking forward to have final fix.

Thanks once again

On Thu, Aug 8, 2019 at 9:41 AM Sebastian Borza notifications@github.com wrote:

Yes, this needs to be fixed. I'm looking into this now...thanks a lot for your patience https://github.com/influxdata/influxdb-python/pull/737!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/influxdata/influxdb-python/issues/738?email_source=notifications&email_token=ADQSV54QKJJATQDCSYZQFHTQDOMHBA5CNFSM4IFVLFZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD32LVSQ#issuecomment-519355082, or mute the thread https://github.com/notifications/unsubscribe-auth/ADQSV56BNEDIYKS6SSMWY4TQDOMHBANCNFSM4IFVLFZQ .

audiofeature commented 5 years ago

I advocate for this to be fixed as well. New user to Influx DB -> very frustrating.

kevmk04 commented 5 years ago

Same issue... was struggling to figure out why it wasn't working.

dtosato commented 4 years ago

This issue is somehow closely related with the issue #696, in fact even for that issue it is necessary to downgrade to Pandas 0.23.4. Please fix the issue, otherwise it is not possible to use colab with influx data.

Karan-M-Shah commented 4 years ago

Any updates on this issue? I cannot write a dataframe into influx