Download and parse data from Garmin Connect or a Garmin watch, FitBit CSV, and MS Health CSV files into and analyze data in Sqlite serverless databases with Jupyter notebooks.
GNU General Public License v2.0
1.1k
stars
138
forks
source link
Failed to parse Message DataMessage from WELLNESS fit files #229
Describe the bug
Before I talk about the bug, I just want to say how much I appreciate the work you have done here - even if this issue I'm having is just a "me" problem and I never get it resolved this project is still great, anyway...
When running "garmindb_cli.py --all --download --import --analyze --latest", the files get downloaded without issue and lots of data lands in the various Sqlite databases which is great, however I get lots of error messages similar to the below:
_Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248794252902_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activitytype) : 13 is not a valid ActivityType
It always seems to be ActivityType13, as an aside that may or may not be related within the garmin_monitoring.db, with exception of the _attributes table, all the other tables are empty.
To Reproduce
Steps to reproduce the behavior:
Run garmindb_cli.py --all --download --import --analyze --latest
Observe potential error messages relating to ActivityType 13
Expected behavior
ActivityType 13 to be parsed and included in the database as per all other ActivityTypes
LogsHere is a selection from the log file that I think is relevant
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_message.py", line 68, in init
data_field = DataField(fit_file, definition_message, definition_message.field_definitions[index], measurement_system)
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 26, in init
super().init(file, schema, None, definition_message.endian)
File "/usr/local/lib/python3.10/dist-packages/fitfile/data.py", line 103, in init
self._convert()
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 44, in _convert
raise FitDataFieldParse(self.field_value, self.field, e)
fitfile.exceptions.exceptions.FitDataFieldParse: Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/garmindb/fit_data.py", line 51, in process_files
fit_file = fitfile.file.File(file_name, self.measurement_system)
File "/usr/local/lib/python3.10/dist-packages/fitfile/file.py", line 46, in init
self.parse(file)
File "/usr/local/lib/python3.10/dist-packages/fitfile/file.py", line 71, in parse
data_message = DataMessage(definition_message, file, self.measurement_system, data_message_context)
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_message.py", line 70, in init
raise FitMessageParse(self, e)
fitfile.exceptions.exceptions.FitMessageParse: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
ERROR:/usr/local/lib/python3.10/dist-packages/garmindb/fit_data.py:Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248155628540_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
ERROR:root:Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248155628540_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType - Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 42, in _convert
self.values = self.field.convert(self.field_value, self.field_definition.invalid(), self.measurement_system)
File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 71, in convert
return [FieldValue(self, value, invalid, **{self._name: self._convert_many(value, invalid)})]
File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 66, in _convert_many
return self.__convert_many(self._convert_single, value, invalid)
File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 62, in convert_many
return [_convert_single(sub_value, invalid) for sub_value in value]
File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 62, in
return [_convert_single(sub_value, invalid) for sub_value in value]
File "/usr/local/lib/python3.10/dist-packages/fitfile/activity_fields.py", line 65, in _convert_single
return ActivityType(value)
File "/usr/lib/python3.10/enum.py", line 385, in call
return cls.new(cls, value)
File "/usr/lib/python3.10/enum.py", line 710, in new__
raise ve_exc
ValueError: 13 is not a valid ActivityType
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_message.py", line 68, in init
data_field = DataField(fit_file, definition_message, definition_message.field_definitions[index], measurement_system)
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 26, in init
super().init(file, schema, None, definition_message.endian)
File "/usr/local/lib/python3.10/dist-packages/fitfile/data.py", line 103, in init
self._convert()
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 44, in _convert
raise FitDataFieldParse(self.field_value, self.field, e)
fitfile.exceptions.exceptions.FitDataFieldParse: Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/garmindb/fit_data.py", line 51, in process_files
fit_file = fitfile.file.File(file_name, self.measurement_system)
File "/usr/local/lib/python3.10/dist-packages/fitfile/file.py", line 46, in init
self.parse(file)
File "/usr/local/lib/python3.10/dist-packages/fitfile/file.py", line 71, in parse
data_message = DataMessage(definition_message, file, self.measurement_system, data_message_context)
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_message.py", line 70, in init
raise FitMessageParse(self, e)
fitfile.exceptions.exceptions.FitMessageParse: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
INFO:root:skipping non-matching File(<UnknownEnumValue.UnknownEnumValue_73: 73> /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248232287519_SKIN_TEMP.fit <UnknownEnumValue.UnknownEnumValue_73: 73> [<MessageType.file_id: 0>, <MessageType.file_creator: 49>, <MessageType.device_info: 23>, <UnknownMessageType.unknown_398: 398>] dev fields {})
INFO:root:skipping non-matching File(<FileType.metrics: 44> /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248232249036_METRICS.fit FileType.metrics [<MessageType.file_id: 0>, <MessageType.file_creator: 49>, <MessageType.device_info: 23>, <MessageType.local_time: 241>, <UnknownMessageType.unknown_410: 410>, <MessageType.unknown_284: 284>] dev fields {})
ERROR:/usr/local/lib/python3.10/dist-packages/garmindb/fit_data.py:Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248183741479_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
ERROR:root:Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248183741479_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType - Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 42, in _convert
self.values = self.field.convert(self.field_value, self.field_definition.invalid(), self.measurement_system)
File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 71, in convert
return [FieldValue(self, value, invalid, **{self._name: self._convert_many(value, invalid)})]
File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 66, in _convert_many
return self.__convert_many(self._convert_single, value, invalid)
File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 62, in convert_many
return [_convert_single(sub_value, invalid) for sub_value in value]
File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 62, in
return [_convert_single(sub_value, invalid) for sub_value in value]
File "/usr/local/lib/python3.10/dist-packages/fitfile/activity_fields.py", line 65, in _convert_single
return ActivityType(value)
File "/usr/lib/python3.10/enum.py", line 385, in call
return cls.new(cls, value)
File "/usr/lib/python3.10/enum.py", line 710, in new__
raise ve_exc
ValueError: 13 is not a valid ActivityType
Describe the bug Before I talk about the bug, I just want to say how much I appreciate the work you have done here - even if this issue I'm having is just a "me" problem and I never get it resolved this project is still great, anyway...
When running "garmindb_cli.py --all --download --import --analyze --latest", the files get downloaded without issue and lots of data lands in the various Sqlite databases which is great, however I get lots of error messages similar to the below:
_Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248794252902_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activitytype) : 13 is not a valid ActivityType
It always seems to be ActivityType13, as an aside that may or may not be related within the garmin_monitoring.db, with exception of the _attributes table, all the other tables are empty.
To Reproduce Steps to reproduce the behavior:
Expected behavior ActivityType 13 to be parsed and included in the database as per all other ActivityTypes
Logs Here is a selection from the log file that I think is relevant Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/fitfile/data_message.py", line 68, in init data_field = DataField(fit_file, definition_message, definition_message.field_definitions[index], measurement_system) File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 26, in init super().init(file, schema, None, definition_message.endian) File "/usr/local/lib/python3.10/dist-packages/fitfile/data.py", line 103, in init self._convert() File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 44, in _convert raise FitDataFieldParse(self.field_value, self.field, e) fitfile.exceptions.exceptions.FitDataFieldParse: Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/garmindb/fit_data.py", line 51, in process_files fit_file = fitfile.file.File(file_name, self.measurement_system) File "/usr/local/lib/python3.10/dist-packages/fitfile/file.py", line 46, in init self.parse(file) File "/usr/local/lib/python3.10/dist-packages/fitfile/file.py", line 71, in parse data_message = DataMessage(definition_message, file, self.measurement_system, data_message_context) File "/usr/local/lib/python3.10/dist-packages/fitfile/data_message.py", line 70, in init raise FitMessageParse(self, e) fitfile.exceptions.exceptions.FitMessageParse: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
ERROR:/usr/local/lib/python3.10/dist-packages/garmindb/fit_data.py:Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248155628540_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType ERROR:root:Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248155628540_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType - Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 42, in _convert self.values = self.field.convert(self.field_value, self.field_definition.invalid(), self.measurement_system) File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 71, in convert return [FieldValue(self, value, invalid, **{self._name: self._convert_many(value, invalid)})] File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 66, in _convert_many return self.__convert_many(self._convert_single, value, invalid) File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 62, in convert_many return [_convert_single(sub_value, invalid) for sub_value in value] File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 62, in
return [_convert_single(sub_value, invalid) for sub_value in value]
File "/usr/local/lib/python3.10/dist-packages/fitfile/activity_fields.py", line 65, in _convert_single
return ActivityType(value)
File "/usr/lib/python3.10/enum.py", line 385, in call
return cls.new(cls, value)
File "/usr/lib/python3.10/enum.py", line 710, in new__
raise ve_exc
ValueError: 13 is not a valid ActivityType
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/fitfile/data_message.py", line 68, in init data_field = DataField(fit_file, definition_message, definition_message.field_definitions[index], measurement_system) File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 26, in init super().init(file, schema, None, definition_message.endian) File "/usr/local/lib/python3.10/dist-packages/fitfile/data.py", line 103, in init self._convert() File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 44, in _convert raise FitDataFieldParse(self.field_value, self.field, e) fitfile.exceptions.exceptions.FitDataFieldParse: Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/garmindb/fit_data.py", line 51, in process_files fit_file = fitfile.file.File(file_name, self.measurement_system) File "/usr/local/lib/python3.10/dist-packages/fitfile/file.py", line 46, in init self.parse(file) File "/usr/local/lib/python3.10/dist-packages/fitfile/file.py", line 71, in parse data_message = DataMessage(definition_message, file, self.measurement_system, data_message_context) File "/usr/local/lib/python3.10/dist-packages/fitfile/data_message.py", line 70, in init raise FitMessageParse(self, e) fitfile.exceptions.exceptions.FitMessageParse: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType
INFO:root:skipping non-matching File(<UnknownEnumValue.UnknownEnumValue_73: 73> /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248232287519_SKIN_TEMP.fit <UnknownEnumValue.UnknownEnumValue_73: 73> [<MessageType.file_id: 0>, <MessageType.file_creator: 49>, <MessageType.device_info: 23>, <UnknownMessageType.unknown_398: 398>] dev fields {}) INFO:root:skipping non-matching File(<FileType.metrics: 44> /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248232249036_METRICS.fit FileType.metrics [<MessageType.file_id: 0>, <MessageType.file_creator: 49>, <MessageType.device_info: 23>, <MessageType.local_time: 241>, <UnknownMessageType.unknown_410: 410>, <MessageType.unknown_284: 284>] dev fields {}) ERROR:/usr/local/lib/python3.10/dist-packages/garmindb/fit_data.py:Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248183741479_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType ERROR:root:Failed to parse /mnt/docker01/dockerdata/garmindb_data/FitFiles/Monitoring/2024/248183741479_WELLNESS.fit: Failed to parse Message DataMessage(<MessageType.monitoring_info: 103>: []): Failed to parse DataField [6, 1, 13] for Field ActivityTypeField (activity_type) : 13 is not a valid ActivityType - Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/fitfile/data_field.py", line 42, in _convert self.values = self.field.convert(self.field_value, self.field_definition.invalid(), self.measurement_system) File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 71, in convert return [FieldValue(self, value, invalid, **{self._name: self._convert_many(value, invalid)})] File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 66, in _convert_many return self.__convert_many(self._convert_single, value, invalid) File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 62, in convert_many return [_convert_single(sub_value, invalid) for sub_value in value] File "/usr/local/lib/python3.10/dist-packages/fitfile/fields.py", line 62, in
return [_convert_single(sub_value, invalid) for sub_value in value]
File "/usr/local/lib/python3.10/dist-packages/fitfile/activity_fields.py", line 65, in _convert_single
return ActivityType(value)
File "/usr/lib/python3.10/enum.py", line 385, in call
return cls.new(cls, value)
File "/usr/lib/python3.10/enum.py", line 710, in new__
raise ve_exc
ValueError: 13 is not a valid ActivityType