arpanghosh8453 / public-fitbit-projects

A script to fetch data from Fitbit servers using their API and store the data in a local influxdb database.
BSD 4-Clause "Original" or "Old" License
323 stars 29 forks source link

mitigating ratelimits while setting up environment #18

Open AskAlice opened 2 months ago

AskAlice commented 2 months ago

I think it should first try to connect to influxdb before making any fitbit api calls. No reason to collect the data if it can't report it.

arpanghosh8453 commented 2 months ago

Hey @AskAlice, Your point is valid. The container is designed to pull data on a rolling basis, so if you fix the connection, the next time, it will write properly. Thanks for taking a closer look to the codebase and pointing this out. Perks of open source project I guess. I agree on the fact that there should be a initial check which confirms a valid influxdb connection before starting everything. That will be an easy fix!

arpanghosh8453 commented 2 months ago

I think this is already covered by the following code - (line #169 - #182)

if INFLUXDB_VERSION == "2":
    try:
        influxdbclient = InfluxDBClient2(url=INFLUXDB_URL, token=INFLUXDB_TOKEN, org=INFLUXDB_ORG)
        influxdb_write_api = influxdbclient.write_api(write_options=SYNCHRONOUS)
    except InfluxDBError as err:
        logging.error("Unable to connect with influxdb 2.x database! Aborted")
        raise InfluxDBError("InfluxDB connection failed:" + str(err))
elif INFLUXDB_VERSION == "1":
    try:
        influxdbclient = InfluxDBClient(host=INFLUXDB_HOST, port=INFLUXDB_PORT, username=INFLUXDB_USERNAME, password=INFLUXDB_PASSWORD)
        influxdbclient.switch_database(INFLUXDB_DATABASE)
    except InfluxDBClientError as err:
        logging.error("Unable to connect with influxdb 1.x database! Aborted")
        raise InfluxDBClientError("InfluxDB connection failed:" + str(err))

I can't test for influxdb 2.0+ but I think influxdb 1.8 will get terminated before the fetching starts if the connection fails. This check is done only when the script starts. if not, this can be injected in that place!

Influxdb 2.0+ provides a way to test the connection like this :

from influxdb_client import InfluxDBClient

client = InfluxDBClient(url="http://localhost:8080",
                        token="my-token",
                        org="my-org",)

health = client.health()
if health.status == "pass":
    print("Connection success.")
else:
    print(f"Connection failure: {health.message}!")

client.__del__()

But I am not sure what's the influxdb 1.0 equivalent (or even this health check feature is available or not)

AskAlice commented 2 months ago

I think this is already covered by the following code - (line #169 - #182)

I was using influxdb 2.0, and from what I can tell the tz needs to be set to something other than automatic at the very least