tcgoetz / GarminDB

Download and parse data from Garmin Connect or a Garmin watch, FitBit CSV, and MS Health CSV files into and analyze data in Sqlite serverless databases with Jupyter notebooks.
GNU General Public License v2.0
1.18k stars 142 forks source link

unable to get monitoring prior to past 10 days #64

Closed benbowen closed 4 years ago

benbowen commented 4 years ago

I was able to override the units of measurement thing in my other issue, but I can't seem to write to the database anything prior to the past 10 days or so.

I've tried a million various things and am looking for a new direction to pursue. I have data back to ~2016, but I am only getting June 4th until now in the database.

{
    "credentials": {
        "user"                          : "",
        "password"                      : ""
    },
    "data": {
        "weight_start_date"             : "05/01/2020",
        "sleep_start_date"              : "05/01/2020",
        "rhr_start_date"                : "05/01/2020",
        "monitoring_start_date"         : "05/01/2020",
        "download_days"                 : 2000,
        "download_latest_activities"    : 2000,
        "download_all_activities"       : 2000,
        "download_days_overlap"         : 3
    },
    "copy": {
        "mount_dir"                     : "/Volumes/GARMIN"
    },
    "enabled_stats": {
        "monitoring"                    : true,
        "steps"                         : true,
        "itime"                         : true,
        "sleep"                         : true,
        "rhr"                           : true,
        "weight"                        : true,
        "activities"                    : false
    },
    "course_views": {
        "steps"                         : []
    }
}
Grimler91 commented 4 years ago

I also only get the last 10 days in the garmin_summary.db, but garmin.db contains all my data, from the specified starting date

tcgoetz commented 4 years ago

I've tried a million various things and am looking for a new direction to pursue. I have data back to ~2016, but I am only getting June 4th until now in the database.

First is is the data downloaded? For example, do you files like this?

~/HealthData/FitFiles/Monitoring/2016/daily_summary_2016-12-19.json

for the earliest days you have data for? If not adjust the dates in the JSON you quoted and run make garmin

which runs:

$(TIME) $(PYTHON) garmin.py --all --download --import --analyze

they key is that that make target does not pass --latest to garmin.py so it should download all data from the date in the JSON config.

tcgoetz commented 4 years ago

I also only get the last 10 days in the garmin_summary.db, but garmin.db contains all my data, from the specified starting date

Do you have other types of data but not daily monitoring data? The summary start date is based on the earliest date in the monitoring data. It's controlled from here: https://github.com/tcgoetz/GarminDB/blob/9337bb8dbdeb997a0d2afe3491382c5ff507821f/analyze_garmin.py#L319

Grimler91 commented 4 years ago

@tcgoetz I mv'ed my HealthData folder and re-ran make create_dbs with the develop branch (instead of the master branch as before). This time I got all dates in garmin_summary.db as well. Maybe this was related to the multiple year issue that you fixed in https://github.com/tcgoetz/GarminDB/commit/c0af13d74e9168e2449bbd3dd6aa42ca054ff557.

tcgoetz commented 4 years ago

This time I got all dates in garmin_summary.db as well.

That's great. I have a few more things to clean up and then I will do a merge to master and a new release.

tcgoetz commented 4 years ago

@Grimler91 Would you mind giving me your data for the success stories wiki page? https://github.com/tcgoetz/GarminDB/wiki/Success-Stories

Grimler91 commented 4 years ago

@tcgoetz sure! I have successfully built and used GarminDB from source (develop branch) on:

I have a forerunner 945 without any extra sensors.