Improve login by migrating Cloudscraper to Garth

matin commented 1 year ago

This branch can be installed with: pip install git+https://github.com/matin/python-garminconnect.git@garth#egg=garminconnect

Garth uses the same API as the mobile app (see #141) and closes the following issues in the process:

[x] #113 by @raffa105 - Caused by rate limits from logging in through the website interface
[x] #127 by @BernieV1977 - Also caused by using Cloudscraper and logging in through the website interface
[x] #129 by @dr400 - Multi-Factor Authentication
[x] #135 by @raffa105 - Garth uses its own consumer key and secret, but it's possible to use a custom consumer key and secret
[x] #136 by @cam-rod - Multi-Factor Authentication. This PR would no longer be necessary. Garth's approach generates a MFA token that's valid for a year

This PR also adds a Jupyter Notebook with examples and tests using pytest and VCR to ensure requests are recorded and don't need to be repeated by someone running the tests. The test coverage is only 50%, but I recommend adding more tests in another PR vs trying to improve test coverage in this PR.

@cyberjunky Let me know if this looks good to you or if there are other changes you'd like to see. I deleted your example.py in favor of the Jupyter Notebook and tests as references, but I can add it back in. It's just a matter of updating how authentication is handled.

cyberjunky commented 1 year ago

@matin Sorry to keep you waiting for so long, I had a busy time, and today I reserved the time to migrate yours and others work again!

cyberjunky commented 1 year ago

@matin I get these kind of error running the test, do I overlook something?

pytest --cov=garminconnect --cov-report=term-missing
======================================================================================== test session starts ========================================================================================
platform linux -- Python 3.11.4, pytest-7.4.2, pluggy-1.3.0
rootdir: /home/ron/development/python-garminconnect
plugins: vcr-1.0.2, cov-4.1.0
collected 14 items                                                                                                                                                                                  

tests/test_garmin.py FFFFFFFFFFFFFF                                                                                                                                                           [100%]

============================================================================================= FAILURES ==============================================================================================
____________________________________________________________________________________________ test_stats _____________________________________________________________________________________________

garmin = <garminconnect.Garmin object at 0x7f2f1215c990>

    @pytest.mark.vcr
    def test_stats(garmin):
>       garmin.login()

tests/test_garmin.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
garminconnect/__init__.py:159: in login
    self.garth.login(self.username, self.password)
../../.local/lib/python3.11/site-packages/garth/http.py:138: in login
    self.oauth1_token, self.oauth2_token = sso.login(*args, client=self)
../../.local/lib/python3.11/site-packages/garth/sso.py:54: in login
    client.get("sso", "/sso/embed", params=SSO_EMBED_PARAMS)
../../.local/lib/python3.11/site-packages/garth/http.py:132: in get
    return self.request("GET", *args, **kwargs)
../../.local/lib/python3.11/site-packages/garth/http.py:115: in request
    self.last_resp = self.sess.request(
/usr/lib/python3/dist-packages/requests/sessions.py:589: in request
    resp = self.send(prep, **send_kwargs)
/usr/lib/python3/dist-packages/requests/sessions.py:703: in send
    r = adapter.send(request, **kwargs)
/usr/lib/python3/dist-packages/requests/adapters.py:489: in send
    resp = conn.urlopen(
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:704: in urlopen
    httplib_response = self._make_request(
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:441: in _make_request
    httplib_response = conn.getresponse(buffering=True)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <vcr.patch.VCRRequestsHTTPSConnection/home/ron/development/python-garminconnect/tests/cassettes/test_stats.yaml object at 0x7f2f144ca190>, _ = False, kwargs = {'buffering': True}

    def getresponse(self, _=False, **kwargs):
        """Retrieve the response"""
        # Check to see if the cassette has a response for this request. If so,
        # then return it
        if self.cassette.can_play_response_for(self._vcr_request):
            log.info(f"Playing response for {self._vcr_request} from cassette")
            response = self.cassette.play_response(self._vcr_request)
            return VCRHTTPResponse(response)
        else:
            if self.cassette.write_protected and self.cassette.filter_request(self._vcr_request):
>               raise CannotOverwriteExistingCassetteException(
                    cassette=self.cassette,
                    failed_request=self._vcr_request,
                )
E               vcr.errors.CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('/home/ron/development/python-garminconnect/tests/cassettes/test_stats.yaml') in your current record mode ('none').
E               No match for the request (<Request (GET) https://sso.garmin.com/sso/embed?id=gauth-widget&embedWidget=true&gauthHost=https%3A%2F%2Fsso.garmin.com%2Fsso>) was found.
E               Found 4 similar requests with 3 different matcher(s) :
E               
E               1 - (<Request (GET) https://thegarth.s3.amazonaws.com/oauth_consumer.json>).
E               Matchers succeeded : ['method', 'scheme', 'port']
E               Matchers failed :
E               host - assertion failure :
E               sso.garmin.com != thegarth.s3.amazonaws.com
E               path - assertion failure :
E               /sso/embed != /oauth_consumer.json
E               query - assertion failure :
E               [('embedWidget', 'true'), ('gauthHost', 'https://sso.garmin.com/sso'), ('id', 'gauth-widget')] != []
E               
E               2 - (<Request (GET) https://connectapi.garmin.com/userprofile-service/socialProfile>).
E               Matchers succeeded : ['method', 'scheme', 'port']
E               Matchers failed :
E               host - assertion failure :
E               sso.garmin.com != connectapi.garmin.com
E               path - assertion failure :
E               /sso/embed != /userprofile-service/socialProfile
E               query - assertion failure :
E               [('embedWidget', 'true'), ('gauthHost', 'https://sso.garmin.com/sso'), ('id', 'gauth-widget')] != []
E               
E               3 - (<Request (GET) https://connectapi.garmin.com/userprofile-service/userprofile/user-settings>).
E               Matchers succeeded : ['method', 'scheme', 'port']
E               Matchers failed :
E               host - assertion failure :
E               sso.garmin.com != connectapi.garmin.com
E               path - assertion failure :
E               /sso/embed != /userprofile-service/userprofile/user-settings
E               query - assertion failure :
E               [('embedWidget', 'true'), ('gauthHost', 'https://sso.garmin.com/sso'), ('id', 'gauth-widget')] != []
E               
E               4 - (<Request (GET) https://connectapi.garmin.com/usersummary-service/usersummary/daily/mtamizi?calendarDate=2023-07-01>).
E               Matchers succeeded : ['method', 'scheme', 'port']
E               Matchers failed :
E               host - assertion failure :
E               sso.garmin.com != connectapi.garmin.com
E               path - assertion failure :
E               /sso/embed != /usersummary-service/usersummary/daily/mtamizi
E               query - assertion failure :
E               [('embedWidget', 'true'), ('gauthHost', 'https://sso.garmin.com/sso'), ('id', 'gauth-widget')] != [('calendarDate', '2023-07-01')]

../../.local/lib/python3.11/site-packages/vcr/stubs/__init__.py:263: CannotOverwriteExistingCassetteException
_________________________________________________________________________________________ test_user_summary _________________________________________________________________________________________

I will add the example.py back I think, found it handy, we could have both Jupyter and rest.

cyberjunky commented 1 year ago

I will need to convert my home assistant integration to use the new version. https://github.com/cyberjunky/home-assistant-garmin_connect

matin commented 1 year ago

Amazing to see this merged!

The issue with the tests has to do with the tests attempting to log in because there isn't a session saved. I'll update the README and also provide a way to run the tests without needing an active session.

cyberjunky commented 1 year ago

Ah of course. Yeah i'm quit happy with your work, needed a bit of searching around at first. One thing I didn't get to work yet is a DELETE call using your gart.post (and override) see issue below, maybe you have an idea, or can make a seperate .garth.delete() call like code in that PR, it worked with set_gear_default?, I copied that.. for delete_weigh_ins() but don't know what the cause is yet

https://github.com/cyberjunky/python-garminconnect/pull/132#issuecomment-1719396710

matin commented 1 year ago

I just created #149 for the tests. I'll look into DELETEs

cyberjunky / python-garminconnect

Improve login by migrating Cloudscraper to Garth #144