is anyone else having an issue downloading files?

rsjrny commented 2 years ago

I am getting to the User Stats Done then hanging. I just did a fresh pull of this repo to see if it helps but no luck. The authenticating URL step seems to be taking a long delay.

If I copy the URL and paste it into my browser I am sent to my activities without issue

Connecting to Garmin Connect... Done. [INFO] Requesting Login ticket Requesting Login ticket... Done. Ticket=ST-0774660-2WO9JpRWhWmiyHIwnO6r-cas [INFO] Authentication URL https://connect.garmin.com/modern/activities?ticket=ST-0774660-2WO9JpRWhWmiyHIwnO6r-cas

pe-st commented 2 years ago

For me it works as usual

2022-08-03 21:53:14,525 [INFO   ] Authentication URL https://connect.garmin.com/modern/activities?ticket=ST-01238961-........-cas
2022-08-03 21:53:18,680 [INFO   ] Profile page https://connect.garmin.com/modern/profile

The authentication takes about 4 seconds, which is about the same time that I had in the last couple of weeks (as recorded in my logfile)

rsjrny commented 2 years ago

very strange. I just ran a test with limit 2, the process ran without issue. I upped the limit = 50 and now it just stalls. I let it sit for 5 minutes before canceling.

I changed the limit to 2 again and it works.

I upped the limit= 12 and now it is hung on the connecting to.

I changed the limit to 5 and ran again. this time it skipped the first, already downloaded" and hung on the second download.

Seems to be very random where it hangs

pe-st commented 2 years ago

I can see no influence here on the count. I tried with --count all (more than 1000) and it still works.

have you tried downloading into an new, empty directory?
you could run it in verbose mode -v and posting the contents of the logfile, maybe there would be a hint?

rsjrny commented 2 years ago

A new output folder made no difference. Some days it works perfectly, most days it does not. Below are three attempts today within 5 minutes. Some days it may actually download a few activities before it hangs

I have also doe a fresh pull of the repository as of 10/17/2022

I am getting very different results. One execution may stop after requesting ticket: Connecting to Garmin Connect... Done. [INFO] Requesting Login ticket Requesting Login ticket... Done. Ticket=ST-0326285-FOjDBZK2Vr2CnJzledqY-cas [INFO] Authentication URL https://connect.garmin.com/modern/activities?ticket=ST-0326285-FOjDBZK2Vr2CnJzledqY-cas

Then I cancel the command window and execute it again, now it habgs at another command location:

Connecting to Garmin Connect... Done. [INFO] Requesting Login ticket Requesting Login ticket... Done. Ticket=ST-0330635-xesyZpw4eBc6V0IoKHim-cas [INFO] Authentication URL https://connect.garmin.com/modern/activities?ticket=ST-0330635-xesyZpw4eBc6V0IoKHim-cas Authenticating... Done. [INFO] Profile page https://connect.garmin.com/modern/profile

immediately run it again and it hangs at authentication:

Connecting to Garmin Connect... Done. [INFO] Requesting Login ticket Requesting Login ticket... Done. Ticket=ST-0328006-ATlqgrRmeT0opIfScI2p-cas [INFO] Authentication URL https://connect.garmin.com/modern/activities?ticket=ST-0328006-ATlqgrRmeT0opIfScI2p-cas

rsjrny commented 2 years ago

it has not worked in 9 days. today I tried it and requested 100 files. The process hung up after 14. I let it set for 2 hours before cancelling.

`Connecting to Garmin Connect... Done. [INFO] Requesting Login ticket Requesting Login ticket... Done. Ticket=ST-0929723-cxNykdqnsvdsyXS96pf5-cas [INFO] Authentication URL https://connect.garmin.com/modern/activities?ticket=ST-0929723-cxNykdqnsvdsyXS96pf5-cas Authenticating... Done. [INFO] Profile page https://connect.garmin.com/modern/profile Getting display name... Done. displayName=rsjrny [INFO] Userstats page https://connect.garmin.com/modern/proxy/userstats-service/statistics/rsjrny Fetching user stats... Done. [INFO] Activity list URL https://connect.garmin.com/modern/proxy/activitylist-service/activities/search/activities?start=0&limit=100 Querying list of activities 1..100... Done. Downloading: Garmin Connect activity (1/100) [9837424496] Shepherdstown - ER9 (Endurance Run) 2022-10-22T08:30:20+20:00, 02:39:28, 21.118 km Original data file already exists; skipping... Downloading: Garmin Connect activity (2/100) [9824398976] Winchester - R5K2 (5K Pace Intervals) 2022-10-20T05:00:41+20:00, 00:54:29, 5.612 km Original data file already exists; skipping... Downloading: Garmin Connect activity (3/100) [9811830500] Winchester - RF29 (Foundation Run) 2022-10-18T05:01:12+20:00, 00:54:02, 5.043 km Original data file already exists; skipping... Downloading: Garmin Connect activity (4/100) [9795266908] Harrisonburg Running 2022-10-15T10:05:50+20:00, 00:48:34, 3.892 km Original data file already exists; skipping... Downloading: Garmin Connect activity (5/100) [9794797581] Harrisonburg Incident Detected 2022-10-15T09:44:53+20:00, 00:03:27, 0.038 km Original data file already exists; skipping... Downloading: Garmin Connect activity (6/100) [9794781627] Harrisonburg Half Marathon 2022-10-15T07:12:09+20:00, 02:25:33, 20.750 km Downloading: Garmin Connect activity (7/100) [9790984382] Frederick County Running 2022-10-14T17:22:16+20:00, 00:10:10, 0.977 km Downloading: Garmin Connect activity (8/100) [9781958865] Winchester Running 2022-10-13T04:42:08+20:00, 01:05:27, 6.579 km Downloading: Garmin Connect activity (9/100) [9773728996] Cardio 2022-10-11T15:07:14+20:00, 00:41:06, 0.000 km Downloading: Garmin Connect activity (10/100) [9770039347] Winchester - OUR8 (Over/Under Intervals Run) 2022-10-11T05:02:50+20:00, 00:53:23, 6.791 km Downloading: Garmin Connect activity (11/100) [9766633800] Cardio 2022-10-10T14:44:55+20:00, 01:02:01, 0.000 km Downloading: Garmin Connect activity (12/100) [9758942747] Army 10 Miler 2022-10-09T08:38:44+20:00, 01:50:18, 16.442 km Downloading: Garmin Connect activity (13/100) [9752158137] Clarke County - RD4 (Depletion Run) 2022-10-08T07:40:43+20:00, 02:26:15, 16.142 km Downloading: Garmin Connect activity (14/100) [9742943346] Frederick County Running

`

pe-st commented 2 years ago

@rsjrny I'm sorry to read that you have problems downloading your data.

I see that you started the script with the verbose flag; can you look in the download folder for the gcexport.log file and paste the last couple of lines? Maybe there is some information pointing at your problem, but maybe it is also a question of the internet connection. Are you using a mobile connection, ADSL or fibre optics? Is it a fast connection?

In my case these lines look like:

2022-10-27 06:28:20,320 [DEBUG  ] Got 200 in 0.44195740699999675 s from https://connect.garmin.com/modern/proxy/activity-service/activity/9742309254
2022-10-27 06:28:20,320 [DEBUG  ] Headers returned:
Date: Thu, 27 Oct 2022 04:28:20 GMT
Content-Type: application/json;charset=UTF-8
Content-Length: 2921
Connection: close
Server-Timing: cf-q-config;dur=6.0000002122251e-06
Server: cloudflare
(rest of the headers omitted)

2022-10-27 06:28:20,550 [DEBUG  ] Got 200 in 0.22709458500000324 s from https://connect.garmin.com/modern/proxy/gear-service/gear/filterGear?activityId=9742309254
2022-10-27 06:28:20,551 [DEBUG  ] Headers returned:
(omitted)

2022-10-27 06:28:20,775 [DEBUG  ] Got 200 in 0.22121194300000013 s from https://connect.garmin.com/modern/proxy/activity-service/activity/9722554756
2022-10-27 06:28:20,776 [DEBUG  ] Headers returned:
(omitted)

rsjrny commented 2 years ago

here is the latest log. I am using a fiber connection through Verizon

2022-10-26 18:16:41,718 [INFO   ] Profile page https://connect.garmin.com/modern/profile
2022-10-26 18:16:42,171 [DEBUG  ] Got 200 in 0.46515289996750653 s from https://connect.garmin.com/modern/profile
2022-10-26 18:16:42,171 [DEBUG  ] Headers returned:
Date: Wed, 26 Oct 2022 22:16:40 GMT
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Connection: close
cache-control: no-cache, no-store, max-age=0, must-revalidate
content-language: en-US
expires: 0
pragma: no-cache
set-cookie: ADRUM_BTa=R:0|g:e920d59f-29be-4102-aa58-abc94a349d3e|n:garmin_869629ee-d273-481d-b5a4-f4b0a8c4d5a3; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/; Secure
set-cookie: ADRUM_BT1=R:0|i:259038; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/; Secure
set-cookie: ADRUM_BTa=R:0|g:3988db80-a942-4c2d-8682-8c8fe4145bfd; Max-Age=30; Expires=Wed, 26-Oct-2022 22:17:10 GMT; Path=/; Secure
set-cookie: ADRUM_BTa=R:0|g:3988db80-a942-4c2d-8682-8c8fe4145bfd|n:garmin_869629ee-d273-481d-b5a4-f4b0a8c4d5a3; Max-Age=30; Expires=Wed, 26-Oct-2022 22:17:10 GMT; Path=/; Secure
set-cookie: SameSite=None; Max-Age=30; Expires=Wed, 26-Oct-2022 22:17:10 GMT; Path=/; Secure
set-cookie: ADRUM_BT1=R:0|i:259028; Max-Age=30; Expires=Wed, 26-Oct-2022 22:17:10 GMT; Path=/; Secure
set-cookie: ADRUM_BT1=R:0|i:259028|e:29; Max-Age=30; Expires=Wed, 26-Oct-2022 22:17:10 GMT; Path=/; Secure
strict-transport-security: max-age=31536000 ; includeSubDomains
x-content-type-options: nosniff
x-frame-options: DENY
x-vcap-request-id: cf655d05-8587-46fe-6de7-49fc338e109e
x-xss-protection: 1; mode=block
CF-Cache-Status: DYNAMIC
Server-Timing: cf-q-config;dur=6.0000020312145e-06
Server: cloudflare
CF-RAY: 76068ec5cf53067c-IAD

2022-10-26 18:16:42,187 [INFO   ] Userstats page https://connect.garmin.com/modern/proxy/userstats-service/statistics/rsjrny

pe-st commented 2 years ago

It looks like Garmin connect doesn't return you the userstats document. In my case I get an answer in less than a second:

2022-10-28 22:22:15,806 [INFO   ] Userstats page https://connect.garmin.com/modern/proxy/userstats-service/statistics/eschep
2022-10-28 22:22:16,080 [DEBUG  ] Got 200 in 0.2738845750000003 s from https://connect.garmin.com/modern/proxy/userstats-service/statistics/eschep

When you visit your profile page in the Browser (https://connect.garmin.com/modern/profile/rsjrny), does the Lifetime Totals section display correctly? (it contains the same information as the Userstats page).

rsjrny commented 2 years ago

it does act like a connection issue but i see no dropped connections in my event log. Some days it will go as far as downloading a couple of files and other days it hangs on "Userstats" or "getting display name". It is very random where it hangs up.

I have other processes that require a connection and I do not have an issue with those, one does updates from the internet every 1 second.

pe-st / garmin-connect-export

is anyone else having an issue downloading files? #82