kjkjava / garmin-connect-export

Download a copy of your Garmin Connect data, including stats and GPX tracks.
MIT License
314 stars 133 forks source link

Unexpected HTTP error (503) #18

Open tymmej opened 8 years ago

tymmej commented 8 years ago

Hi,

I'm using latest version of script. I get raise Exception('Failed. Got an unexpected HTTP error (' + str(e.code) + ').') Exception: Failed. Got an unexpected HTTP error (503). when trying to run script. Last week everything was working.

I tried putting url (url_gc_gpx_activity + a['activity']['activityId'] + '?full=true) and I get The requested endpoint is retired

URL for downloading through browser is https://connect.garmin.com/modern/proxy/download-service/export/gpx/activity/12691xxxxxx

kjkjava commented 8 years ago

Uh oh! The URL you provided works for me (replacing the xxxxxx, of course). To clarify, does it work for you, too?

tymmej commented 8 years ago

The url used by script (http://connect.garmin.com/proxy/activity-service-1.1/gpx/activity/) does not work.

frechina commented 8 years ago

@tymmej you're right. It does not work.

any idea about how to fix it?

tymmej commented 8 years ago

Using same url as browser? I haven't tried that and don't know if there's any difference in .gpx files.

frechina commented 8 years ago

I tried but it does not work. it seems that some info in the cookie is needed.

Maybe @kjkjava can help us

tymmej commented 8 years ago

Or User-agent

frechina commented 8 years ago

@tymmej @kjkjava I partially fixed the issue by:

However, I'm getting the following error message when getting some tcx (I don't know why) {"message":"java.util.ArrayList cannot be cast to java.util.Map","error":"ClassCastException"}

ebrensi commented 8 years ago

The endpoint http://connect.garmin.com/proxy/activity-service-1.1/gpx/activity has been "retired" (deprecated) apparently.

If what you want is GIS coordinate data, you can get that directly as JSON using their activityDetails endpoint, https://connect.garmin.com/modern/proxy/activity-service-1.3/json/activityDetails/{activityID},

which is more efficient than downloading a GPX file and parsing it anyway.

tymmej commented 8 years ago

@frechina can you share patch?

frechina commented 8 years ago

@tymmej I think @ebrensi approach is the best way to fix this. Maybe @ebrensi can share with us how to parse the activityDetails json

tymmej commented 7 years ago

@frechina GPX has points every second, JSON every ~40. And I want to have GPX files. It's easier to share them with others (various online visualizers, import to Strava or Endomondo...). Also in case I will resign from using Garmin Connect having GPX files will be better.

ebrensi commented 7 years ago

@tymmej your comment about data point frequency is interesting. I assumed the points would be the same regardless of file format, and never bothered to look. Can someone confirm this?

I got the idea for using the json endpoint from Tapiriik, in this file https://github.com/cpfair/tapiriik/blob/master/tapiriik/services/GarminConnect/garminconnect.py

tymmej commented 7 years ago

@ebrensi JSON: "metrics" : [ 0.0, 51.13505808636546, 0.0, 25.0, 116.0, 0.0, 2.0000000298023223E-4, 1.469850536E12, 4.204799938201905, 14.269406605265925, 16.974758729338646, 0.0 ] }, { "metrics" : [ 1.0, 51.13502682186663, 1.0, 25.0, 116.0, 1.0, 0.0034200000762939454, 1.469850537E12, 5.202000188827515, 11.534024958488798, 16.974790580570698, 0.0 ] }, { "metrics" : [ 157.0, 51.13100937567651, 115.0, 23.0, 116.5999984741211, 157.0, 0.5354600219726563, 1.469850693E12, 19.670400810241702, 3.0502682985880014, 16.974083986133337, 63.0 ] }, { "metrics" : [ 311.0, 51.124709118157625, 269.0, 22.0, 114.5999984741211, 311.0, 1.4050400390625, 1.469850847E12, 22.406399917602542, 2.677806351428362, 16.977131897583604, 63.0 ] }, { "metrics" : [ 465.0, 51.11847021616995, 419.0, 21.0, 124.19999694824219, 465.0, 2.24947998046875, 1.469851001E12, 20.736000823974614, 2.893518404119131, 16.9703334197402, 59.0 ] }, { "metrics" : [ 619.0, 51.110126450657845, 573.0, 20.0, 118.4000015258789, 619.0, 3.2041201171875, 1.469851155E12, 22.62239971160889, 2.6522385236262296, 16.96826552040875, 0.0 ] }, { "metrics" : [ 773.0, 51.10211938619614, 727.0, 20.0, 114.4000015258789, 773.0, 4.1405400390625, 1.469851309E12, 20.45520057678223, 2.9332393875474043, 16.972166541963816, 56.0 ] }, {

JSON with ?full=true at and of url "metrics" : [ { "metrics" : [ 0.0, 51.13505808636546, 0.0, 25.0, 116.0, 0.0, 2.0000000298023223E-4, 1.469850536E12, 4.204799938201905, 14.269406605265925, 16.974758729338646, 0.0 ] }, { "metrics" : [ 1.0, 51.13502682186663, 1.0, 25.0, 116.0, 1.0, 0.0034200000762939454, 1.469850537E12, 5.202000188827515, 11.534024958488798, 16.974790580570698, 0.0 ] }, { "metrics" : [ 157.0, 51.13100937567651, 115.0, 23.0, 116.5999984741211, 157.0, 0.5354600219726563, 1.469850693E12, 19.670400810241702, 3.0502682985880014, 16.974083986133337, 63.0 ] }, { "metrics" : [ 311.0, 51.124709118157625, 269.0, 22.0, 114.5999984741211, 311.0, 1.4050400390625, 1.469850847E12, 22.406399917602542, 2.677806351428362, 16.977131897583604, 63.0 ] }, { "metrics" : [ 465.0, 51.11847021616995, 419.0, 21.0, 124.19999694824219, 465.0, 2.24947998046875, 1.469851001E12, 20.736000823974614, 2.893518404119131, 16.9703334197402, 59.0 ] }, { "metrics" : [ 619.0, 51.110126450657845, 573.0, 20.0, 118.4000015258789, 619.0, 3.2041201171875, 1.469851155E12, 22.62239971160889, 2.6522385236262296, 16.96826552040875, 0.0 ] }, {

GPX exported in browser: `

116
    <time>2016-07-30T03:48:56.000Z</time>
    <extensions>
      <ns3:TrackPointExtension>
        <ns3:atemp>25.0</ns3:atemp>
        <ns3:cad>0</ns3:cad>
      </ns3:TrackPointExtension>
    </extensions>
  </trkpt>
  <trkpt lat="51.13502682186663150787353515625" lon="16.974790580570697784423828125">
    <ele>116</ele>
    <time>2016-07-30T03:48:57.000Z</time>
    <extensions>
      <ns3:TrackPointExtension>
        <ns3:atemp>25.0</ns3:atemp>
        <ns3:cad>0</ns3:cad>
      </ns3:TrackPointExtension>
    </extensions>
  </trkpt>
  <trkpt lat="51.13499010913074016571044921875" lon="16.974816061556339263916015625">
    <ele>116</ele>
    <time>2016-07-30T03:48:58.000Z</time>
    <extensions>
      <ns3:TrackPointExtension>
        <ns3:atemp>25.0</ns3:atemp>
        <ns3:cad>0</ns3:cad>
      </ns3:TrackPointExtension>
    </extensions>
  </trkpt>
  <trkpt lat="51.13496638834476470947265625" lon="16.97483357973396778106689453125">
    <ele>116</ele>
    <time>2016-07-30T03:48:59.000Z</time>
    <extensions>
      <ns3:TrackPointExtension>
        <ns3:atemp>25.0</ns3:atemp>
        <ns3:cad>0</ns3:cad>
      </ns3:TrackPointExtension>
    </extensions>
  </trkpt>
  <trkpt lat="51.13494149409234523773193359375" lon="16.97485269047319889068603515625">
    <ele>116</ele>
    <time>2016-07-30T03:49:00.000Z</time>
    <extensions>
      <ns3:TrackPointExtension>
        <ns3:atemp>25.0</ns3:atemp>
        <ns3:cad>0</ns3:cad>
      </ns3:TrackPointExtension>
    </extensions>
  </trkpt>

`

JSON even doesn't have info about date of activity.

Edit: ?maxSize=999999999 to get all points.

thedbexp commented 7 years ago

I managed to make this work using the 1.2 API: https://connect.garmin.com/proxy/activity-service-1.2/rest.activity.activityIdValue.html

I only made a change to the tcx line (I'm importing historical data to Strava) pull request added:

diff --git a/gcexport.py b/gcexport.py index 2c6fec7..2a2bf9a 100755 --- a/gcexport.py +++ b/gcexport.py @@ -92,7 +92,7 @@ url_gc_login = 'https://sso.garmin.com/sso/login?service=https%3A%2F%2Fconne url_gc_post_auth = 'https://connect.garmin.com/post-auth/login?' url_gc_search = 'http://connect.garmin.com/proxy/activity-search-service-1.0/json/activities?' url_gc_gpx_activity = 'http://connect.garmin.com/proxy/activity-service-1.1/gpx/activity/' -url_gc_tcx_activity = 'http://connect.garmin.com/proxy/activity-service-1.1/tcx/activity/' +url_gc_tcx_activity = 'http://connect.garmin.com/proxy/activity-service-1.2/tcx/activity/' url_gc_original_activity = 'http://connect.garmin.com/proxy/download-service/files/activity/'

tymmej commented 7 years ago

@thedbexp That's not resolving for other filetypes: for gpx "endpoint is retired".

jsanpedro commented 7 years ago

GPX download was not working for me, changing the url_gc_gpx_activity to url_gc_gpx_activity = 'https://connect.garmin.com/modern/proxy/download-service/export/gpx/activity/'

solved the issue for me

frechina commented 7 years ago

I'm getting error on this url which gets activities

http://connect.garmin.com/proxy/activity-search-service-1.0/json/activities

guys, does it work for you ?

ebrensi commented 7 years ago

I think Garmin just shut down this endpoint. It could have been because of me. I just applied for access t to the Wellness API through the Garmin developer program (http://developer.garmin.com/wellness-api/overview) and in my application I mentioned my app and the informal API I'm using. Then a few days later it no longer works.

The reply I got from them, by the way, was

Thank you for your interest in the Garmin Developer Program. A publically available data privacy policy is a requirement to enter into the Developer Program. Once you have a publicly available data privacy policy, and are actively engaging end-users, please feel free to reapply at that time. Thanks!

moderation commented 7 years ago

I don't believe the endpoint is down. When in a browser and logged into Garmin Connect I can load https://connect.garmin.com/proxy/activity-search-service-1.2/json/activities?start=0&limit=1 successfully. I think they must have changed the required cookie or auth headers in a way that is breaking the code. I haven't worked it out yet.

frechina commented 7 years ago

@moderation you're right. However I checked all the login process and seems so similar than before

frechina commented 7 years ago

I got it. You need to call these endpoint at the end of the login process

GET 'http://connect.garmin.com/modern'; GET 'https://connect.garmin.com/legacy/session';

moderation commented 7 years ago

@frechina That worked! Amazing. How did you work that out? My heavily modified but working fork is at https://github.com/moderation/garmin-connect-export if anyone is interested.

frechina commented 7 years ago

@moderation I used interceptor (from Postman) to capture all request made during Garmin login process. Then, after inspecting such amount of request I realised that the important ones were /modern and /legacy/session ;)

johannesh83 commented 7 years ago

@moderation Awesome https://github.com/moderation/garmin-connect-export working out of the box. I just needed to replace the old one with yours. Merci for the fork

TobiasGirschick commented 7 years ago

thx a lot!

fjgal commented 7 years ago

Thanks everyone for finding the solution to the issues and implementing the fixes. I created a pull request containing the authentication and activity download fixes (no refactoring). I'm aware there were already forks working but I wanted to see if this could be merged into @kjkjava upstream repo.

Meanwhile you can use clone my fork https://github.com/fjgal/garmin-connect-export

elena321 commented 7 years ago

the service to get details of activity doesn't work any more (The requested endpoint is retired): https://connect.garmin.com/proxy/activity-service-1.3/json/activity/XXXXXXXXXXX Is there any new version of it?

emtoonst commented 7 years ago

Could it be this? https://connect.garmin.com/proxy/activity-service/activity/XXXXXXXX/