Improve handling of "empty" TCX files/workouts

[chadlwilson]

First of all, thanks a lot for this tool, seems to be doing a great job with my data!

When importing my data I do have hundreds of failed API calls being made for so-called "empty files" from Strava API perspective.

So far all of these appear to be STEP_COUNTER imports from Fitbit -> Endomondo I had enabled for a period which produced TCX files which Strava doesn't like (and which I am I quite happy to not have in Strava, since it was noisy anyway, and didn't add much value)

File 2017-12-09 16:29:50.0.tcx upload Failed. with error : The file is empty, <a href="https://strava.zendesk.com/entries/21823834-Uploading-Empty-Files" target="_blank">More Information</a>.

Possibly we could save a lot of wasted API calls by ignoring these or immediately failing them client side if we can work out how to detect?

Example

<?xml version="1.0" encoding="UTF-8"?>
<TrainingCenterDatabase xmlns="http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2"
                        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                        xsi:schemaLocation="http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2 http://www.garmin.com/xmlschemas/TrainingCenterDatabasev2.xsd">
    <Activities>
        <Activity Sport="Other">
            <Id>2017-12-09T16:29:50Z</Id>
            <Lap StartTime="2017-12-09T16:00:00Z">
                <TotalTimeSeconds>16260.0</TotalTimeSeconds>
                <DistanceMeters>17060.0</DistanceMeters>
                <Calories>1177</Calories>
                <Intensity>Active</Intensity>
                <TriggerMethod>Manual</TriggerMethod>
                <Track/>
            </Lap>
        </Activity>
    </Activities>
</TrainingCenterDatabase>

[
    {"sport": "STEP_COUNTER"},
    {"source": "IMPORT_FITBIT"},
    {"created_date": "2017-12-09 16:29:50.0"},
    {"start_time": "2017-12-09 16:00:00.0"},
    {"end_time": "2017-12-10 15:59:59.0"},
    {"duration_s": 16260},
    {"distance_km": 17.06},
    {"calories_kcal": 1177},
    {"steps": 14578},
    {"speed_avg_kmh": 3.777121771217712}
]

[fbr-github]

I have the same problem with my swimming workouts. Strava says in "My activities" that it has time=0, distance=0 although it's a proper tcx file where everything is in <TotalTimeSeconds> and <DistanceMeters> of <Lap> tags.

Though I uploaded it by using a self-written script with the command: curl -X POST https://www.strava.com/api/v3/uploads -H "Authorization: Bearer xxxxxxxxxxxxxxxx" -F data_type="tcx" -F file=my.tcx -F activity_type="swim"

Now the question is for me if StravaImporter can handle those swimming or actually files with manually created content? If I look at the issue which chadlwilson has reported, it seems it fails.

[chadlwilson]

I think that's a bit of a different problem, because for the files I am talking about the Strava API completely rejects them; rather than uploading them with missing data.

FWIW, my Swimming entries that uploaded with the tool all have zero values for everything except the calories value. Not sure if that's an issue with this tool or just the way Strava/Endomondo import/generate the TCX files for these manual types of activities.

[fbr-github]

I can confirm what chadlwilson reported 1 comment above about swimming. Just the calories value is shown when I upload it with the StravaImporter.

But I have hoped StravaImporter is better than my approach of the curl command written 2 comments above where it adds at least that 1 lap with the distance but without time. But the issue is also that I cannot see time and distance in the "My activities" list after uploading the tcx file. Here's an example file I extracted from my Endomondo backup. Maybe it helps for fixing the issue:

<?xml version="1.0" encoding="UTF-8"?>
<TrainingCenterDatabase
    xmlns="http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2 http://www.garmin.com/xmlschemas/TrainingCenterDatabasev2.xsd">
    <Activities>
        <Activity Sport="swimming">
            <Id>2020-03-05T21:27:13Z</Id>
            <Lap StartTime="2020-03-05T19:01:30Z">
                <TotalTimeSeconds>4500.0</TotalTimeSeconds>
                <DistanceMeters>2250.0</DistanceMeters>
                <Calories>613</Calories>
                <Intensity>Active</Intensity>
                <TriggerMethod>Manual</TriggerMethod>
                <Track>
                    <Trackpoint>
                        <Time>2020-03-05T19:01:30Z</Time>
                        <DistanceMeters>2250.0</DistanceMeters>
                    </Trackpoint>
                </Track>
            </Lap>
        </Activity>
    </Activities>
</TrainingCenterDatabase>

[fishion]

Worth having a look at the documentation here : https://developers.strava.com/docs/uploads/

Strava reads TCX Courses and only uses the , and from the tag. All other averaging information is ignored.

In your case @chadlwilson I guess it's seeing the empty node and rejecting it as empty. Should hopefully be easy enough to read and parse the XML and verify that there is data in that node before trying to upload. As you say, the end result shouldn't be any different but would save API calls and so process large datasets faster.

In your case @fbr-github, I may have to try it out to see in more detail what's happening, but uploading via curl vs using the tool should produce basically the same results. I'm pretty sure they're ultimately using the same APIs under the hood.

[fbr-github]

In your case @fbr-github, I may have to try it out to see in more detail what's happening, but uploading via curl vs using the tool should produce basically the same results. I'm pretty sure they're ultimately using the same APIs under the hood.

Thank you very much, that would be great. I appreciate your tool very much and wish you a Happy New Year! Maybe, this link can help you: http://frustratedprogrammer.blogspot.com/2012/01/generate-swim-tcx-files-for.html , though I manually experimented a bit with that addionally appended "resting" lap of 60s with 0m and it didn't help, either. The bad thing with Strava is that I cannot export swim workouts to tcx, because when I created a manual swimming entry in Strava it says it's not possible to export (at least with my account for free). But this could help to compare a good against a bad tcx file.

[fishion]

Haven't had a chance to look further into this yet @fbr-github - is it still needed?