Closed MOvations closed 4 years ago
tcgoetz
commented
5 years ago Is the code up to date? If not can you update and retry? If it happens with up to date code, can you send me the FIT file it's crashing on? The bottom of garmin.log should have more info including the file that it crashed on.
MOvations
commented
5 years ago Yeah, I create a fresh python 2.7 environment and do a fresh git request with each of the installs. So far I've had to add after I create the environment: sqlalchemy requests python-dateutil enum34 progressbar2 PyInstaller matplotlib
to get things to run on ubuntu correctly
tried looking around for other packages that might need an update, but nothing seems to change outcome. FYI, I've tried updating: logging datetime zipfile2 json progressbar tempfile2 regex
Don't mind sharing the fit file, just don't want to make it public. How should I share?
tcgoetz
commented
5 years ago make deps should install the required Python packages for you.
I did commit a possible fix 2 hours ago: a971167.
I was going to say you could share the FIT file through dropbox or equivalent and then PM me the link, but I see no ability to PM through GitHub.
MOvations
commented
5 years ago Appreciate the quick responses
make deps did make the install easier, but still failed to unzip the fit data, it does seem consistent that it fails on 9 of 1170
Check linkedin for a link to files
See output below:
Processing daily extra data
Processing daily FIT data
0% (9 of 1170) | | Elapsed Time: 0:00:10 ETA: 1:33:53Traceback (most recent call last):
File "garmin.py", line 344, in
tcgoetz
commented
5 years ago Can you look in your garmin.log file and tell me the name of the last FIT file mentioned before the crash?
MOvations
commented
5 years ago the last few lines of the file: INFO:root:No entry handler _write_hrv_entry for known message type <MessageType.hrv: 78> (10823) from /home/mo/HealthData/FitFiles/Monitoring/2019/2726464275.fit: DataMessage: <MessageType.hrv: 78>: time([invalid] ([547, 65535, 65535, 65535, 65535])), INFO:root:Importing /home/mo/HealthData/FitFiles/Monitoring/2019/596183569.fit (2014-09-21 10:36:49) [FileType.activity] with message types: [<MessageType.file_id: 0>, <MessageType.file_creator: 49>, <MessageType.device_info: 23>, <MessageType.record: 20>, <MessageType.lap: 19>, <MessageType.event: 21>, <MessageType.session: 18>, <MessageType.activity: 34>] INFO:root:file_id message: {'product': <GarminProduct.Fenix2: 1967>, 'number': None, 'time_created': datetime.datetime(2014, 9, 21, 10, 36, 49), 'serial_number': 347, 'type': <FileType.activity: 4>, 'manufacturer': <Manufacturer.Garmin: 1>}
tcgoetz
commented
5 years ago It looks like you and one other user are reporting an issue where your daily monitoring data includes an activity file for a workout. Do you use workouts and do they show up on your daily summary?
MOvations
commented
5 years ago So, I rifled through all the activity types I have and it looks like my data goes back to 2011 when I had a Forerunner 305, I now have a Fenix 5X that includes motion tracking and all-day HRM. I see my activities/workouts in my daily feed. They can range from hiking, cycling, skiing, boating, running, to just a cardio workout with no GPS. Some of the older ones show up as uncategorized or untitled.
On the android app, I see things like my all-day heart rate that also highlights my heartrate while in activity, and it appears to be one continuous data source for that purpose. My daily and 7-day dashboards also include activities/workouts.
Haven't spent too much time rooting around the HealthData files that GarminDB makes, but I can if there's something there I should look for
tcgoetz
commented
5 years ago I've made some fixes. Could you run make update to update your code and then rerun the import?
MOvations
commented
5 years ago Updated a few times, but still failed on the same FIT file with every attempt
tcgoetz
commented
5 years ago Can you send me the stack trace from the screen where it failed? (not the end of the log). It should be different since the code there changed. In particular, this line is no longer the same:
current = GarminDB.Activities.get(self.garmin_act_db, activity_id)
MOvations
commented
5 years ago (py27) mo@DaBox:~/Projects/GarminDB$ make create_dbs
python garmin.py --all --download --import --analyze
Geting activities: '/home/mo/HealthData/FitFiles/Activities' (1000)
100% (742 of 742) |#################################################| Elapsed Time: 0:00:00 Time: 0:00:00
unzip_files: /home/mo/HealthData/FitFiles/Activities
100% (32 of 32) |###################################################| Elapsed Time: 0:00:32 Time: 0:00:32
100% (32 of 32) |###################################################| Elapsed Time: 0:00:49 Time: 0:00:49
unzip_files: /home/mo/HealthData/FitFiles/Monitoring/2019
100% (32 of 32) |###################################################| Elapsed Time: 0:00:32 Time: 0:00:32
100% (32 of 32) |###################################################| Elapsed Time: 0:00:31 Time: 0:00:31
100% (32 of 32) |###################################################| Elapsed Time: 0:00:32 Time: 0:00:32
Processing profile data
100% (1 of 1) |#####################################################| Elapsed Time: 0:00:00 Time: 0:00:00
Processing weight data
100% (32 of 32) |###################################################| Elapsed Time: 0:00:00 Time: 0:00:00
Processing daily summary data
100% (32 of 32) |###################################################| Elapsed Time: 0:00:00 Time: 0:00:00
Processing daily extra data
Processing daily FIT data
0% (9 of 1170) | | Elapsed Time: 0:00:10 ETA: 1:30:01Traceback (most recent call last):
File "garmin.py", line 344, in
I don't see the current = GarminDB.Activities.get(self.garmin_act_db, activity_id) this time...
Verify that I was up to date.... (py27) mo@DaBox:~/Projects/GarminDB$ make update git submodule init git submodule update git pull --rebase Already up to date. Current branch master is up to date.
tcgoetz
commented
5 years ago Ok. This line is fixed:
activity['sub_sport'] = sub_sport.name
Hopefully that is the last one. New data files mean learning new things about the FIT format.
MOvations
commented
5 years ago Looks that solved the problem, but unfortunately, there are more...
(py27) mo@DaBox:~/Projects/GarminDB$ make create_dbs
python garmin.py --all --download --import --analyze
Geting activities: '/home/mo/HealthData/FitFiles/Activities' (1000)
100% (742 of 742) |#######################| Elapsed Time: 0:00:00 Time: 0:00:00
unzip_files: /home/mo/HealthData/FitFiles/Activities
100% (32 of 32) |#########################| Elapsed Time: 0:00:32 Time: 0:00:32
100% (32 of 32) |#########################| Elapsed Time: 0:02:12 Time: 0:02:12
unzip_files: /home/mo/HealthData/FitFiles/Monitoring/2019
100% (32 of 32) |#########################| Elapsed Time: 0:00:32 Time: 0:00:32
100% (32 of 32) |#########################| Elapsed Time: 0:00:32 Time: 0:00:32
100% (32 of 32) |#########################| Elapsed Time: 0:00:32 Time: 0:00:32
Processing profile data
100% (1 of 1) |###########################| Elapsed Time: 0:00:00 Time: 0:00:00
Processing weight data
100% (32 of 32) |#########################| Elapsed Time: 0:00:00 Time: 0:00:00
Processing daily summary data
100% (32 of 32) |#########################| Elapsed Time: 0:00:00 Time: 0:00:00
Processing daily extra data
Processing daily FIT data
0% (9 of 1170) | | Elapsed Time: 0:00:10 ETA: 1:29:24 /home/mo/HealthData/FitFiles/Monitoring/2019/596183569.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183569.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183569.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/596183569.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183569.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183569.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183569.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/596183569.fit session invalid subsp ort value: 255
2% (27 of 1170) | | Elapsed Time: 0:01:06 ETA: 0:26:54 Unrecognized device: {'product': <UnknownProduct.UnknownProduct_18: 18>, 'device _index': 5, 'cum_operating_time': None, 'software_version': None, 'timestamp': d atetime.datetime(2016, 4, 23, 13, 24, 21), 'manufacturer': 3767, 'battery_voltag e': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unk nown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, N one, None, None, None, None, None, None, None, None, None, None, None, None, Non e, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_ty pe': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name '
7% (83 of 1170) |# | Elapsed Time: 0:02:09 ETA: 1:27:12 Unrecognized device: {'product': <UnknownProduct.UnknownProduct_12: 12>, 'device _index': 4, 'cum_operating_time': None, 'software_version': '0.00', 'timestamp': datetime.datetime(2016, 7, 12, 19, 19, 14), 'manufacturer': 2392, 'battery_volt age': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'u nknown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None , 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, Non e, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, N one, None, None], 'ant_transmission_type': None, 'serialnumber': None, 'device type': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': Non e, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'na me'
11% (136 of 1170) |## | Elapsed Time: 0:03:22 ETA: 0:34:55 Unrecognized device: {'product': <UnknownProduct.UnknownProduct_12: 12>, 'device _index': 5, 'cum_operating_time': None, 'software_version': '0.00', 'timestamp': datetime.datetime(2016, 6, 29, 6, 42, 1), 'manufacturer': 2392, 'battery_voltag e': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unk nown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, N one, None, None, None, None, None, None, None, None, None, None, None, None, Non e, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_ty pe': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name '
19% (223 of 1170) |#### | Elapsed Time: 0:05:35 ETA: 0:17:09 /home/mo/HealthData/FitFiles/Monitoring/2019/594748910.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748910.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748910.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/594748910.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748910.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748910.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748910.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/594748910.fit session invalid subsp ort value: 255
40% (472 of 1170) |######## | Elapsed Time: 0:10:39 ETA: 0:25:34 /home/mo/HealthData/FitFiles/Monitoring/2019/594748914.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748914.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748914.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/594748914.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748914.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748914.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748914.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/594748914.fit session invalid subsp ort value: 255
56% (660 of 1170) |############ | Elapsed Time: 0:14:51 ETA: 0:17:26 /home/mo/HealthData/FitFiles/Monitoring/2019/596183570.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183570.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183570.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/596183570.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183570.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183570.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183570.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/596183570.fit session invalid subsp ort value: 255
57% (673 of 1170) |############ | Elapsed Time: 0:15:46 ETA: 0:41:53 /home/mo/HealthData/FitFiles/Monitoring/2019/596183565.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183565.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183565.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/596183565.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183565.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183565.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183565.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/596183565.fit session invalid subsp ort value: 255
57% (674 of 1170) |############ | Elapsed Time: 0:16:20 ETA: 4:46:11 /home/mo/HealthData/FitFiles/Monitoring/2019/594748915.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748915.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748915.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/594748915.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748915.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748915.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748915.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/594748915.fit session invalid subsp ort value: 255
61% (721 of 1170) |############# | Elapsed Time: 0:17:40 ETA: 0:09:41 /home/mo/HealthData/FitFiles/Monitoring/2019/596183568.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183568.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183568.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/596183568.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183568.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183568.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183568.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/596183568.fit session invalid subsp ort value: 255
63% (738 of 1170) |############# | Elapsed Time: 0:18:31 ETA: 0:07:37 Unrecognized device: {'product': <UnknownProduct.UnknownProduct_18: 18>, 'device _index': 3, 'cum_operating_time': None, 'software_version': None, 'timestamp': d atetime.datetime(2016, 5, 4, 10, 56, 33), 'manufacturer': 3767, 'battery_voltage ': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unkn own_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, ' battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, No ne, None, None, None, None, None, None, None, None, None, None, None, None, None , None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_typ e': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name'
63% (746 of 1170) |############## | Elapsed Time: 0:18:39 ETA: 0:07:19 /home/mo/HealthData/FitFiles/Monitoring/2019/594748918.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748918.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748918.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/594748918.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748918.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748918.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748918.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/594748918.fit session invalid subsp ort value: 255
72% (843 of 1170) |############### | Elapsed Time: 0:20:17 ETA: 0:02:38 Unrecognized device: {'product': <UnknownProduct.UnknownProduct_18: 18>, 'device _index': 5, 'cum_operating_time': None, 'software_version': '1.95', 'timestamp': datetime.datetime(2016, 4, 16, 11, 19, 4), 'manufacturer': 3767, 'battery_volta ge': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'un known_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None , None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, No ne, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_t ype': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None , 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'nam e'
72% (850 of 1170) |############### | Elapsed Time: 0:20:22 ETA: 0:03:33 /home/mo/HealthData/FitFiles/Monitoring/2019/594748911.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748911.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748911.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/594748911.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748911.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/594748911.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/594748911.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/594748911.fit session invalid subsp ort value: 255
86% (1013 of 1170) |################## | Elapsed Time: 0:24:00 ETA: 0:02:24 /home/mo/HealthData/FitFiles/Monitoring/2019/596183571.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183571.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183571.fit session invalid sport value: Sport.cycling
/home/mo/HealthData/FitFiles/Monitoring/2019/596183571.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183571.fit session invalid sport value: Sport.transition
/home/mo/HealthData/FitFiles/Monitoring/2019/596183571.fit session invalid subsp ort value: 255
/home/mo/HealthData/FitFiles/Monitoring/2019/596183571.fit session invalid sport value: Sport.running
/home/mo/HealthData/FitFiles/Monitoring/2019/596183571.fit session invalid subsp ort value: 255
91% (1073 of 1170) |################### | Elapsed Time: 0:25:50 ETA: 0:01:48 Unrecognized device: {'product': <UnknownProduct.UnknownProduct_12: 12>, 'device _index': 3, 'cum_operating_time': None, 'software_version': '0.00', 'timestamp': datetime.datetime(2016, 7, 2, 12, 4, 57), 'manufacturer': 2392, 'battery_voltag e': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unk nown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, N one, None, None, None, None, None, None, None, None, None, None, None, None, Non e, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_ty pe': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name '
100% (1170 of 1170) |#####################| Elapsed Time: 0:27:40 Time: 0:27:40
Processing sleep data
100% (32 of 32) |#########################| Elapsed Time: 0:00:10 Time: 0:00:10
Processing rhr data
100% (32 of 32) |#########################| Elapsed Time: 0:00:00 Time: 0:00:00
Processing all activities summary data from /home/mo/HealthData/FitFiles/Activit ies
100% (742 of 742) |#######################| Elapsed Time: 0:00:04 Time: 0:00:04
Processing activities detail data
100% (742 of 742) |#######################| Elapsed Time: 0:00:04 Time: 0:00:04
Processing activities extra data
N/A% (0 of 156) | | Elapsed Time: 0:00:00 ETA: --:--:-- Traceback (most recent call last):
File "garmin.py", line 344, in
tcgoetz
commented
5 years ago Looking into it. BTW, you have copies of your activity file sin your monitoring directory.
tcgoetz
commented
5 years ago You have a lot of TCX files that have no start or end time which is not something I have expected. That's the first bug in your last message. Will put out a fix soon.
tcgoetz
commented
5 years ago The second error:
Fit.field_enums.SubSport.invalid:
File "/home/mo/anaconda3/envs/py27/lib/python2.7/site-packages/enum/init.py", line 390, in getattr
raise AttributeError(name)
AttributeError: invalidLooks like your Fit submodule was not updated. Did you do a git pull instead of a make update? 'invalid' is a new member added to Fit.field_enums.SubSport as part of the fix.
tcgoetz
commented
5 years ago commit dd409fcbeecaf5c1465544f79f5d95435d4d6c40 fixes the TCX parsing issue.
Please run make update as this commit updates the tcxparser git submodule. Then run as usual.
MOvations
commented
5 years ago Thanks for all this.
I believe the above traceback was from a full reinstall, but I usually start by doing "make update". I do a reinstall just to double-check
Not sure how I have copies of my activity files in your monitoring directory, is there something I should do to correct this?
Thanks again
MOvations
commented
5 years ago Traceback from the most recent attempt:
(py27) mo@DaBox:~/Projects/GarminDB$ make update git submodule init git submodule update Submodule path 'Fit': checked out 'c2d80f5579784594ae9b05f81769e7b97a0a126b' git pull --rebase remote: Enumerating objects: 24, done. remote: Counting objects: 100% (24/24), done. remote: Compressing objects: 100% (16/16), done. remote: Total 24 (delta 7), reused 20 (delta 7), pack-reused 0 Unpacking objects: 100% (24/24), done. From https://github.com/tcgoetz/GarminDB 0ff03d8..dd409fc master -> origin/master
tcgoetz
commented
5 years ago Sorry that didn't work. It worked for me on your TCX files, so it has to be an environmental difference. I will use a bigger hammer on handling exceptions from the TCX parser. I'll put something out later today after some tests.
tcgoetz
commented
5 years ago Please try again. Run make update first. Thanks!
MOvations
commented
5 years ago Looks like the big hammer worked. I'll tinker more tomorrow. A few files still had an issue, but here's the traceback
(base) mo@DaBox:~$ cd Projects/GarminDB/ && conda activate py27 (py27) mo@DaBox:~/Projects/GarminDB$ make update git submodule init git submodule update git pull --rebase remote: Enumerating objects: 47, done. remote: Counting objects: 100% (47/47), done. remote: Compressing objects: 100% (25/25), done. remote: Total 47 (delta 20), reused 38 (delta 19), pack-reused 0 Unpacking objects: 100% (47/47), done. From https://github.com/tcgoetz/GarminDB dd409fc..63cb15e master -> origin/master Fetching submodule Fit From https://github.com/tcgoetz/Fit 2a05aa1..90b33ab master -> origin/master Updating dd409fc..63cb15e Fast-forward .gitignore | 5 +- Fit | 2 +- GarminDB/activities_db.py | 27 +++- HealthDB/db.py | 54 +++----- analyze_fitbit.py | 6 +- analyze_garmin.py | 14 +-- analyze_mshealth.py | 6 +- fit_file_processor.py | 140 ++++++++++----------- import_garmin.py | 14 ++- import_garmin_activities.py | 30 ++--- tcx_file.py | 6 +- test/Makefile | 6 +- test/test_activities_db.py | 32 ++++- test/test_db_base.py | 28 ++++- test/test_files/json/activity/details/readme.txt | 1 + test/test_files/json/activity/summary/readme.txt | 1 + test/test_files/json/monitoring/summary/readme.txt | 1 + test/test_monitoring_db.py | 26 ++++ 18 files changed, 239 insertions(+), 160 deletions(-) create mode 100644 test/test_files/json/activity/details/readme.txt create mode 100644 test/test_files/json/activity/summary/readme.txt create mode 100644 test/test_files/json/monitoring/summary/readme.txt Current branch master is up to date. (py27) mo@DaBox:~/Projects/GarminDB$ make create_dbs time python garmin.py --all --download --import --analyze Geting activities: '/home/mo/HealthData/FitFiles/Activities' (1000) 100% (742 of 742) |#######################| Elapsed Time: 0:00:00 Time: 0:00:00 unzip_files: /home/mo/HealthData/FitFiles/Activities 100% (32 of 32) |###################################################| Elapsed Time: 0:00:32 Time: 0:00:32 100% (32 of 32) |###################################################| Elapsed Time: 0:02:06 Time: 0:02:06 unzip_files: /home/mo/HealthData/FitFiles/Monitoring/2019 100% (32 of 32) |###################################################| Elapsed Time: 0:00:32 Time: 0:00:32 100% (32 of 32) |###################################################| Elapsed Time: 0:00:32 Time: 0:00:32 100% (32 of 32) |###################################################| Elapsed Time: 0:00:32 Time: 0:00:32 Processing profile data 100% (1 of 1) |#####################################################| Elapsed Time: 0:00:00 Time: 0:00:00 Processing weight data 100% (32 of 32) |###################################################| Elapsed Time: 0:00:00 Time: 0:00:00 Processing daily summary data 100% (32 of 32) |###################################################| Elapsed Time: 0:00:00 Time: 0:00:00 Processing daily extra data Processing daily FIT data 100% (1170 of 1170) |###############################################| Elapsed Time: 0:11:09 Time: 0:11:09 Processing sleep data 100% (32 of 32) |###################################################| Elapsed Time: 0:00:06 Time: 0:00:06 Processing rhr data 100% (32 of 32) |###################################################| Elapsed Time: 0:00:00 Time: 0:00:00 Processing all activities summary data from /home/mo/HealthData/FitFiles/Activities 100% (742 of 742) |#################################################| Elapsed Time: 0:00:04 Time: 0:00:04 Processing activities detail data 100% (742 of 742) |#################################################| Elapsed Time: 0:00:04 Time: 0:00:04 Processing activities extra data 100% (156 of 156) |#################################################| Elapsed Time: 0:00:01 Time: 0:00:01 Processing activities FIT data 2% (16 of 586) |# | Elapsed Time: 0:00:46 ETA: 0:06:45Unrecognized device: {'product': <UnknownProduct.UnknownProduct_18: 18>, 'device_index': 5, 'cum_operating_time': None, 'software_version': None, 'timestamp': datetime.datetime(2016, 4, 23, 13, 24, 21), 'manufacturer': 3767, 'battery_voltage': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unknown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_type': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name' 6% (40 of 586) |### | Elapsed Time: 0:01:29 ETA: 0:35:34Unrecognized device: {'product': <UnknownProduct.UnknownProduct_12: 12>, 'device_index': 4, 'cum_operating_time': None, 'software_version': '0.00', 'timestamp': datetime.datetime(2016, 7, 12, 19, 19, 14), 'manufacturer': 2392, 'battery_voltage': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unknown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_type': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name' 10% (63 of 586) |##### | Elapsed Time: 0:02:21 ETA: 0:12:29Unrecognized device: {'product': <UnknownProduct.UnknownProduct_12: 12>, 'device_index': 5, 'cum_operating_time': None, 'software_version': '0.00', 'timestamp': datetime.datetime(2016, 6, 29, 6, 42, 1), 'manufacturer': 2392, 'battery_voltage': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unknown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_type': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name' 65% (382 of 586) |############################### | Elapsed Time: 0:12:57 ETA: 0:09:08Unrecognized device: {'product': <UnknownProduct.UnknownProduct_18: 18>, 'device_index': 3, 'cum_operating_time': None, 'software_version': None, 'timestamp': datetime.datetime(2016, 5, 4, 10, 56, 33), 'manufacturer': 3767, 'battery_voltage': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unknown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_type': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name' 72% (424 of 586) |################################### | Elapsed Time: 0:14:10 ETA: 0:04:50Unrecognized device: {'product': <UnknownProduct.UnknownProduct_18: 18>, 'device_index': 5, 'cum_operating_time': None, 'software_version': '1.95', 'timestamp': datetime.datetime(2016, 4, 16, 11, 19, 4), 'manufacturer': 3767, 'battery_voltage': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unknown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_type': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name' 91% (537 of 586) |############################################ | Elapsed Time: 0:18:01 ETA: 0:02:07Unrecognized device: {'product': <UnknownProduct.UnknownProduct_12: 12>, 'device_index': 3, 'cum_operating_time': None, 'software_version': '0.00', 'timestamp': datetime.datetime(2016, 7, 2, 12, 4, 57), 'manufacturer': 2392, 'battery_voltage': None, 'ant_related': None, 'hardware_version': None, 'unknown_8': None, 'unknown_9': None, 'source_type': <SourceType.local: 5>, 'ant_device_number': None, 'battery_status': <BatteryStatus.invalid: 255>, 'unknown_17': [None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None], 'ant_transmission_type': None, 'serial_number': None, 'device_type': <LocalDeviceType.accelerometer: 3>, 'unknown_23': None, 'unknown_16': None, 'ant_network': <AntNetwork.invalid: 255>} - 'int' object has no attribute 'name' 100% (586 of 586) |#################################################| Elapsed Time: 0:19:19 Time: 0:19:19 Summary Table Generation Generating 2018 100% (1 of 1) |#####################################################| Elapsed Time: 0:00:00 Time: 0:00:00 100% (52 of 52) |###################################################| Elapsed Time: 0:00:01 Time: 0:00:01 100% (1 of 1) |#####################################################| Elapsed Time: 0:00:00 Time: 0:00:00 Generating 2019 100% (33 of 33) |###################################################| Elapsed Time: 0:00:10 Time: 0:00:10 100% (52 of 52) |###################################################| Elapsed Time: 0:00:01 Time: 0:00:01 100% (2 of 2) |#####################################################| Elapsed Time: 0:00:00 Time: 0:00:00 1839.28user 15.43system 35:20.41elapsed 87%CPU (0avgtext+0avgdata 426616maxresident)k 0inputs+651800outputs (0major+1506558minor)pagefaults 0swaps python mshealth.py python fitbit.py Years (0): []
tcgoetz
commented
5 years ago make update to get commit 805455d04fc5c5345a9e0e10c47454e6846d0926 and the 'int' object has no attribute 'name' errors should be fixed.
There are other fixes and speed improvements too.
tcgoetz
commented
4 years ago Are you all set now? Can I close this issue?
MOvations
commented
4 years ago Sorry, been busy. It works, feel free to close. Did a full reinstall, worked w/o error. Not sure if it was faster, ran it in the background. Worked correctly.
Some notes on my install procedure, as "make deps" didn't still didn't solve my packages issues:
conda create -n py27 python=2.7 sqlalchemy requests python-dateutil enum34 progressbar2 PyInstaller matplotlib conda activate py27 git clone https://github.com/tcgoetz/GarminDB.git && pip install python-tcxparser cd GarminDB sudo nano GarminConnectConfig.json.example # -->open, add creds, save as GarminConnectConfig.json make setup make create_dbs
tcgoetz
commented
4 years ago I'm not surprised that make deps isn't compatible with Conda. I think conda always wants to use it's own package manager and not pip.
If you have any thoughts on visualizations for GarminDB data, let me know. You can file them as a new issue on the project.
Whacked the local HealthData files and rebuilt the environment twice now but it keeps dying when I unzipping the files after I download the 742 activities from Garmin. Here's a sample of the output of the error: Probably need to install/update a package. Thoughts?
Processing daily FIT data 0% (9 of 1170) | | Elapsed Time: 0:00:10 ETA: 1:33:11Traceback (most recent call last): File "garmin.py", line 344, in
main(sys.argv[1:])
File "garmin.py", line 337, in main
import_data(debug, test, latest, weight, monitoring, sleep, rhr, activities)
File "garmin.py", line 162, in import_data
gfd.process_files(db_params_dict)
File "/home/mo/Projects/GarminDB/import_garmin.py", line 91, in process_files
fp.write_file(Fit.file.File(file_name, self.measurement_system))
File "/home/mo/Projects/GarminDB/Fit/file.py", line 42, in init
self.parse()
File "/home/mo/Projects/GarminDB/Fit/file.py", line 72, in parse
data_message = DataMessage(definition_message, self.file, self.measurement_system)
File "/home/mo/Projects/GarminDB/Fit/data_message.py", line 31, in init
data_field = DataField(fit_file, definition_message, definition_message.field_definitions[index], measurement_system)
File "/home/mo/Projects/GarminDB/Fit/data_field.py", line 23, in init
super(DataField, self).init(file, schema, None, definition_message.endian)
File "/home/mo/Projects/GarminDB/Fit/data.py", line 103, in init
self._convert()
File "/home/mo/Projects/GarminDB/Fit/data_field.py", line 26, in _convert
self.value_obj = self.field.convert(self.dict[self.field.name], self.field_definition.invalid(), self.measurement_system)
File "/home/mo/Projects/GarminDB/Fit/fields.py", line 124, in convert
value_obj = self.obj_func(value / self.scale, invalid)
TypeError: 'int' object is not callable
Makefile:125: recipe for target 'garmin' failed
make: *** [garmin] Error 1