tcgoetz / GarminDB

Download and parse data from Garmin Connect or a Garmin watch, FitBit CSV, and MS Health CSV files into and analyze data in Sqlite serverless databases with Jupyter notebooks.
GNU General Public License v2.0
1.17k stars 142 forks source link

No more VO2_max in steps_activities table since 2024-05-23 #236

Open lpierron opened 4 months ago

lpierron commented 4 months ago

No more VO2_max in steps_activities table since 2024-05-23

Open garmin_activities database with sqlite3.

Log example:

sqlite> select strftime("%Y-%m-%d", start_time) AS day, sport, VO2_Max  from steps_activities JOIN activities ON activities.activity_id = steps_activities.activity_id where start_time > '2024-05-01' ORDER BY start_time DESC;
2024-06-26|running|
2024-06-25|running|
2024-06-22|walking|
2024-06-22|walking|
2024-06-21|running|
2024-06-19|running|
2024-06-17|running|
2024-06-14|running|
2024-06-12|running|
2024-06-11|running|
2024-06-09|walking|
2024-06-09|walking|
2024-06-07|running|
2024-06-05|running|
2024-06-03|running|
2024-05-29|running|
2024-05-28|running|
2024-05-27|running|
2024-05-25|walking|
2024-05-24|running|
2024-05-24|walking|
2024-05-24|running|
2024-05-23|running|
2024-05-22|running|38.0
2024-05-17|running|38.0
2024-05-16|running|38.0
2024-05-14|running|38.0
2024-05-13|running|38.0
2024-05-13|running|
2024-05-06|running|38.0
2024-05-03|running|38.0
2024-05-02|running|38.0
2024-05-01|walking|

You can see that all running and walking activities since 2024-05-23 have no VO2_Max data.

lpierron commented 3 months ago

To help, I wrote a little program to get the VO2Max on my Garmin Connect account with garth: vo2max.py.zip

Partial result of running VO2max.py:

-------------------- api.get_max_metrics('2024-07-31') --------------------
[
    {
        "userId": 50448457,
        "generic": {
            "calendarDate": "2024-07-31",
            "vo2MaxPreciseValue": 38.0,
            "vo2MaxValue": 38.0,
            "fitnessAge": 52,
            "fitnessAgeDescription": null,
            "maxMetCategory": 0
        },
        "cycling": null,
        "heatAltitudeAcclimation": null
    }
]
---------------------------------------------------------------------------
-------------------- api.get_max_metrics('2024-08-01') --------------------
[]
lpierron commented 1 month ago

I found the problem and solution.

Since May 2024, Garmin JSON activity files do not export some fields with a null value like avgVerticalOscillation.

So in this program, the variable avg_vertical_oscillation is set to None and an exception occurs at line #119: https://github.com/tcgoetz/GarminDB/blob/e52ebc2f7f1e92d4607d718af5db961ffedea2f9/garmindb/garmin_json_data.py#L108-L123

So I suggest to modify _get_field_obj in idbutils/json_file_processor.py: https://github.com/tcgoetz/utilities/blob/5ce3f712f34818cddf5327cd827a480989ff6940/idbutils/json_file_processor.py#L79-L84

and replacing by:

    def _get_field_obj(self, json, fieldname, format_func):
        try:
            data = json[fieldname]
            return format_func(data)
        except KeyError as e:
            self.logger.debug("JSON %s not found in %r, setting to `0`: %s", fieldname, json, e)
        return format_func(0)