Bugazelle / export-csv-to-influx

The python library to write the CSV data into Influx. Support me via Bitcoin: bc1qqgkmph9cvygzxfpupv4jr4n0nfx3qumwg39j5w
BSD 3-Clause "New" or "Revised" License
35 stars 10 forks source link

Problem importing csv TypeError: strptime() argument 1 must be string, not int #17

Closed petersg closed 4 years ago

petersg commented 4 years ago

Good morning im trying to import a csv but im having some type of problem.

This is my csv Structure:

id,ida,timestamp,latitude,longitude,altitude,speed,vspeed,quawk,track,ground,leg,date 0,fr24-34530E-1517587275,1517587275,44.224740000000004,9.47696,26725,462,-960,6463,78,False,ANE8786_34530E_1517580840_1517588845,2018-02-02

I dont know where im failing, i hope you can help me! Thanks!

Bugazelle commented 4 years ago

Sorry for the delay. I am focusing on it now. And will response you ASAP.

Bugazelle commented 4 years ago

Hello @petersg,

I found the cause. Your column "timestamp" should be the datetime, something like "2019-07-11 02:04:05". The current is 1517587275, not the datetime. And you could use the --time_format match your specific time format.

Actually, here I used the "timestamp" column, it is a bad name. Now, I am working a new feature to support your requests.

And will let your know once it is ready.

Thank you

petersg commented 4 years ago

Yes my timestamp it's in milliseconds I thought it was option format supported that timestamp ! Thanks for the awnser

Bugazelle commented 4 years ago

Hello @petersg,

You may install & use the v0.1.19 which support the pure timestamp. Please let me know if it is working on your side. You are very welcome if you have any questions.

1. Export Run the following export.

export_csv_to_influx \
    --csv test.csv \
    --dbname demo \
    --measurement test \
    --tag_columns speed,ground \
    --field_columns id,ida,latitude,longitude,altitude,speed,vspeed,quawk,track,ground,leg,date \
    --user admin \
    --password admin \
    --force_insert_even_csv_no_update True \
    --server 127.0.0.1:8086
# Info: Database demo already exists
# Info: Read 1 lines from ./test.csv
# Info: Inserting 1 data_points...
# Info: Wrote 1, response: True
# Info: Done

2. Data in influx

> select * from test
name: test
time             altitude date       ground ground_1 id ida                    latitude           leg                                  longitude quawk speed speed_1 track vspeed
----             -------- ----       ------ -------- -- ---                    --------           ---                                  --------- ----- ----- ------- ----- ------
1517587275000000 26725    2018-02-02 False  False    0  fr24-34530E-1517587275 44.224740000000004 ANE8786_34530E_1517580840_1517588845 9.47696   6463  462   462     78    -960
>
petersg commented 4 years ago

Men! Perfect! Works for me :D Thank you!