tcgoetz / GarminDB

Download and parse data from Garmin Connect or a Garmin watch, FitBit CSV, and MS Health CSV files into and analyze data in Sqlite serverless databases with Jupyter notebooks.
GNU General Public License v2.0
1.11k stars 137 forks source link

Activity Name missing when importing historic activities #131

Closed hirael closed 1 year ago

hirael commented 3 years ago

Hi

I have been a Garmin user since 2014, so i have a large amount of data to import and when importing my historic activities, something happend in 2016 that stopped the Activity Name being imported into the database. I am currently running GarminDB on Windows the command: "python garmindb_cli.py --activities --download --overwrite --import"

I have tried re-downloading the activities several times but they always end up with 675 entries with no name. I have attached the log file and the first Fit file where the import stops. I can send more Fit file examples if needed.

Any guidance would be appreciated - thank you :)

image

garmindb.log activity_details_1378660053.zip

Preliator-code commented 3 years ago

Have you found a solution? For me, I am systematically obliged to delete the "HealthData" folder, and run a garmindb_cli.py --all --download --import --analyze to be able to have the updated descriptions

hirael commented 3 years ago

For my historic missing data I used Garmin Export to download and import the files into a CSV file and copied the info over from there.

tcgoetz commented 2 years ago

I would guess that activities prior to2016 store the name in a different field and the logic that converts activity JSON to DB would need to handle both fields.

hirael commented 2 years ago

From what I could see, "elapsedDuration" was missing in the historic activities....

This change in import_activities.py worked for me:

if self._get_field(json_data, 'elapsedDuration'):
     elapsed_time = fitfile.conversions.secs_to_dt_time(self._get_field(json_data, 'elapsedDuration', int))
 else:
     elapsed_time = fitfile.conversions.secs_to_dt_time(self._get_field(json_data, 'duration', int))

and this change for my non-distance based activities ....

'distance' : distance.kms_or_miles(self.measurement_system) if distance is not None else None,

I did try to report the changes using a pull request, but not sure I got it right!

schroeer commented 1 year ago

I have run into the same problem. The change occured some time between 2016/12/05 and 2016/12/10.

For the older activities, elapsedDuration is present in both files activity_\d*\.json and activity_details_\d*\.json. However, in activity_\d*\.json the value is null. The value in activity_details_\d*\.json is correct.

This currently makes this line fail with TypeError: unsupported operand type(s) for +: 'datetime.datetime' and 'NoneType' when parsing the summary file, as reported in the log. This prevents those fields that are only extracted from the summary file (here) from being recorded. Remarkably, the correct value for elapsed_time from the details file is already currently written to the database, because it belongs to the fields that are extracted from both files (here).

So I believe that a check for the value being numerical should be sufficient.

tcgoetz commented 1 year ago

@schroeer can you propose the change in a pull request?

schroeer commented 1 year ago

I am using the release version and currently don't have a suitable development platform. Consequently, I am able to propose this very simple change but I am not able to test it.

schroeer commented 1 year ago

I have patched my local installation and that works as intended.