Feature/countnew

[chs8691]

Added argument 'count new'. This will download all new activities. For this, a pickle file '.settings' in the download directory will be used. It holds the index of the newest/latest activity that is downloaded successfully. The file will be updated every time an newer activity is downloaded (independent of the arguments). 'Count new ' is using the total activity index. This will fail, if activities will be deleted or inserted on Garmin Connect. To achieve a seamless and stable download history, the download order is now inverted - from oldest to newest. This does not quite fit with the known behavior and in special with the -sa argument. But from my point of view it is the better way. Parameter start_activity_no is still proper working but now a little bit hard to understand :0|

Examples: // User has 5 activities on Garmin Connect with index 1 (oldest) .. index 5 (newest) gcexport count 3 // Fetch activities by index 3-5, save '5' in .settings // User creates new activity with index 6 on Garmin Connect gcexport count new // Fetch activity with index 6 and update .settings with '6' gcexport count new // No effect gcexport count 5 -sa 4 // Fetch activity index 1, .settings will be untouched rm download/.setting gcexport count new // Fetch activities with index 1..6 and update .settings with '6' // User deletes activity with index 3 and creates new activity on Garmin Connect. This new activity will have index 6. gcexport count new // No effect - total number of activities hasn't changed and the last index is still '6' :0(

[bxsx]

I don't see it that useful but I can imagine a lot of additional effort to maintain it later. If I understand it correctly the only difference here is to have ability to keep correct counting even if user broke counting by deleting/inserting activities. I don't think this app should fix user problems. In fact, I'm sure that every user of this script is aware of the limited capabilities and definitely if manually editing Garmin Connect activities then double check expected output from the script. I might also add that the fit files are so small, that simpler way to fix this issue is by removing the directory and re-download the whole history.

[cristian5th]

Hello.

This proposal is very useful in order to avoid downloading the data of all activities when you actually only need the two or three newest ones.

I have created the same functionality at my branch called "personal-adjustments". My branch cannot be merged with the original project because this is not the only modification I have introduced. But it might help you to have a look at how I did it.

Instead of downloading the activities by number, I download the activities by date. This is completely avoiding all the issues you are presenting when activities are being deleted at Garmin Connect. Basically, when the argument is "count new", I change the search params to search_params = {'startDate': last_date, 'sortBy': 'startLocal', 'sortOrder': 'asc', 'limit': num_to_download} and the value of startDate is the date when the last downloaded activity was recorded. This is the data that I store in the file ".settings". Doing like this, if activities are being deleted at Garmin Connect, I will still not miss any new activity at the next download.

Regards.

[bxsx]

I am not against the feature, I am against the current implementation. This introduce a new file and breaks scriptability of the main tool which power was thanks to single file functionality. Introducing new file raises a lot of questions. Where to store it? Shouldn't be in the $HOME/.garmin-connect-export/ directory? What about multiple accounts? I don't think calling it ".settings" is proper as in fact its more likely a database. This raises another questions such as why not use SQLite or whatever else for further performance and functionalities? Etc. etc.

I think the most important part of this tool is to keep connections to GC API stable. There are plenty of small features that could be added and then used by few users. However, this gains complexity of the code and usability in some occasions too.

As I said previously:

I'm sure that every user of this script is aware of the limited capabilities and definitely if manually editing Garmin Connect activities then double check expected output from the script. I might also add that the fit files are so small, that simpler way to fix this issue is by removing the directory and re-download the whole history.

Also, I think this is super rare situation to edit GC history manually.

So, to keep everyone happy, you may try to implement it by adding a new arguments to the command line. I guess a switcher of how to count activities (with the default value to the current default) and maybe some extra controllers.

[pe-st]

If I understood correctly, your proposing to make 3 changes, maybe this should be individual pull requests?:

• changing the order the activities are processed
• adding a new option to download only activities not downloaded
• introducing a local file to remember the last download activity

I can see that for new a reversed order is simpler. But for the number option reversing the counting may make the script more difficult to understand, as the number of activities in the user stats page (the total_to_download) can be misleading. For my account there are about a dozen activities that are not accounted for, let's say I have total_to_download == 1000, but the download loop only downloads 988 activities. You would then have to take into account these missing activities for the -sa option.

Thus for the reversed order I'd prefer it configurable, and maybe force to oldest-first for the new option. If the exporter is integrated into other scripts it might break them if you change the order also for the all and number options.

As for a local file to remember the last downloads.

• why are you using the pickle format? It's not human readable and the script has already other configuration files in the simple properties format
• I'm not sure the download folder is the best place for this file, this makes the new option useless without using always the same -d option (with the default for the export directory a new pickle file is used each day). Probably the name/path of this file should be configurable, with a default in the directory of the script itself
• using .settings as (default) name for this file seems unlucky. Isn't this the name of the settings file of the Eclipse IDE? What about .gcexport? When having an option for name/path you could then also put it into something like $HOME/.garmin-connect-export/, but I personally dislike every little script claiming a dot-directory in the users HOME...
• I think storing the activity date instead of the activity index (as proposed by @cristian5th) would a more stable solution.

[cristian5th]

Hello again.

I just wanted to continue a little more on the way I do it. Please, don't forget that it was a different Christian ( @chs8691 ) who originally proposed this and all credits should go to him.

• I name the .settings file just as simple as last_activity.txt and in there, there is just the date of the latest downloaded activity. The activities from that date will be tried to be downloaded at the next execution of the script and this is done so I ensure that I won't miss any new activities created the same day, before and after the script was executed.
• As to where to store it, I do it at the downloads directory, and why not? There are already many other (I say) unwanted files like all the xml files containing information about the devices. So I don't see harm in keeping this file also there. But I'm open to other suggestions.
• When the script is executed with the new option and the last_activity.txt file is missing, then the last_date is becoming 1901-01-01 and the part of the script that is being executed is the same as for the all option.
• Once the new option is being used, other options like sa are becoming useless and must be ignored.

The main reason I implemented this for myself is because executing the all option everyday is loading the GC servers without need and Garmin people might not like it and close our access one day. Using the number at the count argument is not automated for me because I don't know how many new activities I would have created since the last time the script was executed. So creating a new option was the logical way to do it. The script is automatically executed every day at 3:33 inside a Raspberry Pi at the garage.

Regards.

[chs8691]

Hi! Sorry for my late answer and thank you for the really useful discussion. I agree to your objections and would like to dismiss this request. Nonetheless I still need a 'load new' function. Maybe I'll start a new request in the future, with a better integrated and more stable coding. Happy new year!

[bxsx]

@chs8691 thanks for your effort! I think I have something like “load new” feature in one of my local branches. I will be back to this branch next month and prepare a pull request, so you may find it interesting. I will try to mention you once the pull request will be ready.