Unable to import a file from the combine command on Garmin Connect

[ravenfeld]

Hello, I am trying to combine FIT files (fitactivity --combine -o result.fit part1.fit part2.fit) to import into Garmin Connect a single file. The problem is that when I import I always get a problem from Garmin. If I don't combine them I don't have the problem. I therefore assume that it is the file generated that does not correspond to Garmin. Do you know why?

[muktihari]

Did you use the latest version of fitactivity (v0.21.2) (try fitactivity -v) or did you build it from latest commit from master branch? If the answer is yes, here is what I could say.

It's hard to say without having the files to debug since Garmin Connect (GC) does not tells us the reason why it rejects it.

I encountered similar problem before, in my case, it turned out that my files contains Split Summary messages and when they that have the same split_type it needs to be combined, having two Split Summary that have the same split_type type will make the resulting file marked as invalid by GC. It has been fixed in latest version (v0.21.2), so it maybe different problem.

There are many messages with different way how to handle them, unless we know what's in our files, it's hard to make a workaround. I need to look at your files to be able to assist you, if you don't want to upload the files publicly, you can email me at muktihaz+fit@gmail.com.

[muktihari]

The quick fix maybe we can make a new flag to ignore the optional messages and only include the required messages declared in this specification: https://developer.garmin.com/fit/file-types/activity.

[ravenfeld]

I used version v0.21.2 and as it didn't work I used master for the same result.

Here are the files I'm trying to merge. I don't think it's a weight problem because even when I only do 2 out of 3 Garmin doesn't accept.

https://drive.google.com/drive/folders/11WIsz2FFZHW34poNqsFhoLBdGOrU1IjT?usp=sharing

[muktihari]

I see, let me review the files.

[ravenfeld]

Thank you very much. Will I be able to see the graphs as for a ‘normal’ activity?

[muktihari]

I manage to combine the 3 files and upload to GC by removing unknown messages. Even though GC returns error, the file is successfully uploaded. It takes minutes to be able to see the activity after the error and I found it to be unreliable, the page seems to could not be loaded properly as well, previously it can be loaded but now it stucks like this:

I think this is way past the GC limit (a file must be <25mb or < 99.999poins (72hours if one second record option)), reference: File Size Limitation of Manual Uploads to Garmin Connect

Here is the resulting file if you want to try to upload to GC: https://drive.google.com/drive/folders/1wQGAlprR9kZ-vuI9UOK1A-rOuzh576SP?usp=sharing

I'm sorry I don't have a solution for this right now, it takes so long to test uploading to GC and GC might return unpredictable result. My suggestion for next ultra run is maybe to turn-on smart recording on the smartwatch, so the file can be smaller (?)

[ravenfeld]

Thank you very much, the problem is that I clicked too quickly on save and not on resume later. The problem is on my side. Can I do a douglas pecker to reduce the points?

What if I put the file in my watch so that it's my watch that sends it?

My girlfriend didn't have to worry about back-ups and synchronised with her watch without a hitch.

[ravenfeld]

For your information, Garmin Connect can't display the track but Strava can and yet it's Garmin that sends it to them ;) There are no graphics, is this linked to the combination?

I don't know the GO but maybe make an option when combining to activate a douglasPecker (https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm) to reduce the number of points if that is the problem for some.

If you are interested I have the code in kotlin

[muktihari]

Happy to help @ravenfeld

Can I do a douglas pecker to reduce the points?

I didn’t know this algorithm before but it seems promising to reduce the points and we might found new lead regarding this matter by having fewer points. I’ll take a look once I got more spare time. Thanks for referencing it!

If you are interested I have the code in kotlin

Sure, I would be happy to have a code reference, I did some kotlin in the past so I hopefully can still read it :D

What if I put the file in my watch so that it's my watch that sends it?

I think having the file being uploaded means we somehow manage to pass the “wall”, I assume it is the Garmin Connect UI that is unable to display it due to its limitations.

For your information, Garmin Connect can't display the track but Strava can and yet it's Garmin that sends it to them ;) There are no graphics, is this linked to the combination?

Does the map and graph look good on Strava such as Heart Rate graph, Cadence graph etc? Or it didn’t show at all?

[ravenfeld]

On strava I have the elevation and pace graph, which I don't have on Garmin Connect.

I'm going to test it using the watch to see if it works.

[muktihari]

I have update the gdrive link with new file that has been reduced using Douglas Peucker. Total messages is reduced from 757.248 to 498.208 and Garmin Connect accepts it just fine:

[ravenfeld]

It's perfect, although I think your douglas pecker is a bit too aggressive. It cuts the bends a lot, so the track won't be usable if some people want to look at it. Thank you very much for your time. Maybe put an option on combine so people can use this algo.

[muktihari]

The previous experiment was to see if Garmin Connect can accept the file if the points is smaller. When I try zooming the map, I can't really see the different at glance. I tried reducing 757.248 to 649.995 by adjusting the epsilon but Garmin Connect UI can't display it. Yes, the epsilon will be configurable, thanks for the suggestion.

[ravenfeld]

OK, thank you very much. We'll have to find the maximum number that Garmin Connect can take, but in any case, good work.

[muktihari]

Hi @ravenfeld, let me conclude the issue:

1. The combined file is unable to be uploaded to Garmin Connect, based on my experiment, is because it has many points that Garmin Connect is unable to handle. I was able to upload and it was properly displayed by having fewer points.
2. I have updated the CLI to include reduce feature, it has 3 methods to choose:
   • Based on GPS points using RDP (ref: https://github.com/muktihari/carto)
   • Based on distance interval
   • Based on time interval

The master branch has been updated, so you can try build it from there. Please note that the structure of CLI commands are a bit different, README.md has been updated to accommodate it.

Here is the example for running the program:

• Using RDP:

  fitactivity combine reduce -o result.fit --rdp 0.001 part1.fit part2.fit part3.fit

• Using distance interval (in meters, e.g.: 0.5m):

  fitactivity combine reduce -o result.fit --distance 0.5 part1.fit part2.fit part3.fit

• Using time interval (in seconds, e.g.: 2s):

  fitactivity combine reduce -o result.fit --time 2 part1.fit part2.fit part3.fit

Fyi, you can view the resulting file using my other project https://openivity.github.io, it runs 100% on client-side. It has limitation but I think it is sufficient for simply debugging the map after being reduced.

Anw, I will appreciate any feedback regarding the CLI update. Hopefully it helps.

[ravenfeld]

It's really great what you've done. It suits me perfectly. I'll let you close the exit.