incremental backup won't try to download non-existent FIT files for manually-uploaded activities

[dlenski]

@petergardfjall, This should resolve the issue that you raised at https://github.com/petergardfjall/garminexport/pull/1#issuecomment-75364555

incremental_backup.pygarminbackup.py will no longer try to export FIT files for activities that are missing them because they are stationary/manually-uploaded:

• I've borrowed the approach of Tapiriik, which uses the absence of both sumSampleCountSpeed and sumSampleCountTimestamp to distinguish "stationary" activities (anything that doesn't have some kind of GPS or time data). https://github.com/cpfair/tapiriik/blob/master/tapiriik/services/GarminConnect/garminconnect.py#L292
• I hope this doesn't inadvertently exclude any kinds of stationary activities which could be exported to FIT. I don't have a device capable of generating FIT files for stationary activities, so I can't really test it.

By the way, I had to make one other change, to avoid errors in the case where the activity zip archive doesn't actually contain the expected .fit file. All of my activities were uploaded in TCX format, and the ZIP files simply don't contain FIT versions for most of them (most of them, but not all of them... truly puzzling). Anyway, get_activity_fit now returns None if there's no FIT file in the archive, rather than raising an error.

[petergardfjall]

Great stuff, thanks a lot Dan!

I haven't had time yet to go through the code in detail, and before merging I'd like to convince myself that the "stationary-activity condition" is a sufficient condition for determining if an activity cannot be exported to FIT format.

Do you, for instance, know how it would work in case an activity (with GPS data) has been uploaded in a different format (such as TCX/GPX)? These would not be stationary activities, but would they still be possible to export to FIT format?

I've also considered a (more foolproof?) solution that, on every backup run, would mark every failed .FIT download attempt (as indicated by a 404 - Not Found error code), for example, by appending the missing fit activity file to a .missing_fit_activities file in the backup directory. All fit files listed in .missing_fit_activities would then be considered already backed up (or rather, not in need of being backed up) by the backup script.

So, hopefully I'll be able to put aside some time to look into this sometime later this week. I'll get back to you.

Thank you very much for the time and effort!

[dlenski]

… before merging I'd like to convince myself that the "stationary-activity condition" is a sufficient condition for determining if an activity cannot be exported to FIT format.

Agreed. I'm just not sure how to test edge cases for this.

Do you, for instance, know how it would work in case an activity (with GPS data) has been uploaded in a different format (such as TCX/GPX)? These would not be stationary activities, but would they still be possible to export to FIT format?

Yes, this is what I was referring to above.

Most of my activities were created using a Garmin 405 and uploaded using Garmin's ANT Agent under Windows, which uploads them in TCX format. For most but not all all of the activities that were uploaded in TCX format, the download-service endpoint returns a zip containing a TCX file but no FIT file. In other words it seems that download-service returns a file in whatever format the user originally uploaded. (Take a look at publicly-visible activityId=669692070 as an example.)

It is frustrating that GC has a mechanism to dowload a TCX or GPX representation for any activity but can only handle FIT for activities that were originally uploaded in this format. Although, from the point of view of a "backup" application, saving the original file in its originally-uploaded form is actually a pretty ideal behavior.

Thank you very much for the time and effort!

Thanks for writing this program. I have several years' worth of history on GC and realized I'd like to have a local copy :)

[petergardfjall]

Hi Dan!

I've finally managed to get this done. After giving this some thought I went for the route I described in my last message (that is, marking attempted but failed downloads in a .not_found file), which felt like a safer route, given that I know too little about the mysterious ways of Garmin Connect.

But I did include some of the excellent code you provided. For instance, the code for downloading the original activity file and fit file (I actually thought that the "original activity" always was a .fit file, but you enlightened me there!).

Thanks a lot for the contribution!