Login rate limit - Part II

[sasadangelo]

Hi @cyberjunky,

I just started using gaeminconnect library and it works quite well. However, as documented here: Login rate limit

during my tests I got the following problem:

Traceback (most recent call last):
  File "/Users/sasadangelo/github.com/sasadangelo/hrv/update.py", line 38, in <module>
    main()
  File "/Users/sasadangelo/github.com/sasadangelo/hrv/update.py", line 15, in main
    client.login()
  File "/Users/sasadangelo/github.com/sasadangelo/hrv/venv/lib/python3.12/site-packages/garminconnect/__init__.py", line 204, in login
    self.garth.login(
  File "/Users/sasadangelo/github.com/sasadangelo/hrv/venv/lib/python3.12/site-packages/garth/http.py", line 160, in login
    self.oauth1_token, self.oauth2_token = sso.login(
                                           ^^^^^^^^^^
  File "/Users/sasadangelo/github.com/sasadangelo/hrv/venv/lib/python3.12/site-packages/garth/sso.py", line 84, in login
    client.post(
  File "/Users/sasadangelo/github.com/sasadangelo/hrv/venv/lib/python3.12/site-packages/garth/http.py", line 151, in post
    return self.request("POST", *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sasadangelo/github.com/sasadangelo/hrv/venv/lib/python3.12/site-packages/garth/http.py", line 141, in request
    raise GarthHTTPError(
garth.exc.GarthHTTPError: Error in request: 429 Client Error: Too Many Requests for url: https://sso.garmin.com/sso/signin?id=gauth-widget&embedWidget=true&gauthHost=https%3A%2F%2Fsso.garmin.com%2Fsso%2Fembed&service=https%3A%2F%2Fsso.garmin.com%2Fsso%2Fembed&source=https%3A%2F%2Fsso.garmin.com%2Fsso%2Fembed&redirectAfterAccountLoginUrl=https%3A%2F%2Fsso.garmin.com%2Fsso%2Fembed&redirectAfterAccountCreationUrl=https%3A%2F%2Fsso.garmin.com%2Fsso%2Fembed

I suspected there were some limitations in API usage.

Looking at the previous issue I understand that in 1 hour the block is removed. This was true almost 1 year ago. Is this still valid? What is the approach followed by garminconnect to get the data? Does it scrape HTML data "illegally" or use something legal? I want to avoid that my account is blocked with all my activities and I cannot access them anymore even with the "legal" app. I have a long history of data (almost 10 years).

Once my code is finished I think I will access the data once a day via github actions to update my csv files. However, during the development I need to run several tests. Is there a way to persist the session and avoid to login everytime?

Looking at the code it uses garth to access the data. What it is not clear to me is that garth is another Garmin interface for the Garmin API, correct? If so, why should I use garminconnect? Are there any additional features in this package?

Thank you in advance for your help.

[cyberjunky]

@sasadangelo After we switched using the garth code from @matin we got away with this issue. I haven't run into the limit myself since then, with the current code, but it's not guaranteed... Just try it. I will close issue, simply (re)open when you run into this later using current code.

[matin]

It could also be the IP address. See this discussion:

https://github.com/matin/garth/discussions/60

[sasadangelo]

Hi,

I don't think my public IP is shared. I have Wind operator in Italy and searching a bit I didn't find this issue.

[sasadangelo]

Hi @cyberjunky

I am unable to reopen the issue. However, we can keep it closed if you, guys, reply to my doubts. I tried again and it worked, however, I am afraid it happen again. I think the blocked is removed after 1 hour probably (I tried 1 day later).

However, I reached the too many request in this way. I have an update.py script that download some data in csv files. If I run it multiple times I reach a point where the 409 problem occurs. I don't think it is a problem of shared IP because otherwise it would fail from the first run. Instead, it happen after several run (the number is not quite high).

I think this makes perfectly sense because it is a common practice to limit the number of login attempt, for the same account, in a period of time. The question is: what is this limit?

Moreover, usually client app should run some precautions to avoid to tun the login at every run. Usually, they keep the login module in a session until it expires. Usually, a refresh token is used to keep the session longer.

The point here is: are there any of these precautions I can use for my script?

Once in "production" I am sure I will login once a day, however, during the coding phase I need to run several tests and I need to know what is the limit and how to overcome it.

[cyberjunky]

@sasadangelo Might not be related to your issue at all, but in your updated script, make sure you still use the tokens to authenticate, not your username/login, only use them to once to get and store the tokens to use for further auth, they are valid for 1Y.

[sasadangelo]

Hi @cyberjunky,

Thank you for your reply. I think this is the issue. However, a token that last 1 year it's not the max from a security point of view. Usually, this stuff is managed with token/refresh token that keep the session opened for a while. However, I'll try it. Is there any example I can follow to get the token and store it? I don't see anything in the main README file.

[cyberjunky]

@sasadangelo I think this is what the mobile app does by default, you are right, it's uncommon. But the whole Gamin Connect API is a story by itself... The example.py file uses the 'tokenstore' file by default to login, when doing init_api() if it fails because the tokenstore file isn't available or it's content is invalid (AuthError) it asks for a login/password to retieve it and store it again. https://github.com/cyberjunky/python-garminconnect/blob/2e99a5bbfa69a4fbafd152a35df502308fe2b4d9/example.py#L182

[sasadangelo]

Ok I understand how the process works. I need an empty file like thi ~/.garminconnect. Then if I try to connect using the code here:

tokenstore = os.getenv("GARMINTOKENS") or "~/.garminconnect"
garmin = Garmin()
garmin.login(tokenstore)

the first time I'll get an exception and, in this case I use this code to authenticate va user/password and store the token for the next use:

garmin = Garmin(email=email, password=password, is_cn=False, prompt_mfa=get_mfa)
garmin.login()
garmin.garth.dump(tokenstore)
token_base64 = garmin.garth.dumps()
dir_path = os.path.expanduser(tokenstore_base64)
with open(dir_path, "w") as token_file:
    token_file.write(token_base64)

Am I correct? If so, I think that from a security perspective it's not a good thing. Encode64 is easily revertable, it's not like an encryption.

However, I know this is not your fault but how Garmin API are designed (token of 1 year without refresh token is a bad idea). I understand that from a script perspective the only way to reuse the token is stored it somewhere. I can delete it once I finished my tests.

Thank you very much for your support. If my understanding is correct we can close the issue.

[cyberjunky]

Yeah correct, simply delete it after the time period you feel comfortable with.

[sasadangelo]

Hi @cyberjunky

This is just to thank you for the support. Have a nice day.

[sasadangelo]

Hi again @cyberjunky,

just tested the code in examples.py and there is something bad. I have the files in ~/.garminconnect empty:

(venv) salvatores-mbp:hrv sasadangelo$ cat ~/.garminconnect/oauth
oauth1_token.json  oauth2_token.json  
(venv) salvatores-mbp:hrv sasadangelo$ cat ~/.garminconnect/oauth1_token.json 
(venv) salvatores-mbp:hrv sasadangelo$ cat ~/.garminconnect/oauth2_token.json 

I think that this code has something bad:

    except (FileNotFoundError, GarthHTTPError, GarminConnectAuthenticationError):
        # Session is expired. You'll need to log in again
        print(
            "Login tokens not present, login with your Garmin Connect credentials to generate them.\n"
            f"They will be stored in '{tokenstore}' for future use.\n"
        )
        try:
            # Ask for credentials if not set as environment variables
            if not email or not password:
                email, password = get_credentials()

            garmin = Garmin(email=email, password=password, is_cn=False, prompt_mfa=get_mfa)
            garmin.login()
            # Save Oauth1 and Oauth2 token files to directory for next login
            garmin.garth.dump(tokenstore)
            print(
                f"Oauth tokens stored in '{tokenstore}' directory for future use. (first method)\n"
            )
            # Encode Oauth1 and Oauth2 tokens to base64 string and safe to file for next login (alternative way)
            token_base64 = garmin.garth.dumps()
            dir_path = os.path.expanduser(tokenstore_base64)
            with open(dir_path, "w") as token_file:
                token_file.write(token_base64)
            print(
                f"Oauth tokens encoded as base64 string and saved to '{dir_path}' file for future use. (second method)\n"
            )

I mean. I have the two files in ~/.garminconnect empty while the base64 store contains a value. However, for the next login it use the token in the ~/.garminconnect folders. Please can you clarify how things should work?

Does this line dump in the file:

garmin.garth.dump(tokenstore)

I ask this because if so, it doesn't work. Please can you check your examples.py file to be sure it is coded as you expect?

[cyberjunky]

I think you should not create the file, it gets created, otherwise it maybe use it... Not at the code right now, so can't see exactly, just delete the files, The example file works, so just copy the init part mostly

[cyberjunky]

And you need the try: except: part as well