tcgoetz / GarminDB

Download and parse data from Garmin Connect or a Garmin watch, FitBit CSV, and MS Health CSV files into and analyze data in Sqlite serverless databases with Jupyter notebooks.
GNU General Public License v2.0
1.18k stars 142 forks source link

Import Error #46

Closed AYellowDart closed 4 years ago

AYellowDart commented 4 years ago

Happens when trying to build the database for the first time:

Processing FileType.activity FIT data from /Users//HealthData/FitFiles/Activities
 18% (45 of 242) |####                   | Elapsed Time: 0:01:04 ETA:   0:07:07Traceback (most recent call last):
  File "sqlalchemy/sql/sqltypes.py", line 1479, in _db_value_for_elem
KeyError: 213

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "sqlalchemy/engine/base.py", line 1173, in _execute_context
  File "sqlalchemy/engine/default.py", line 799, in _init_compiled
  File "sqlalchemy/sql/sqltypes.py", line 1584, in process
  File "sqlalchemy/sql/sqltypes.py", line 1495, in _db_value_for_elem
LookupError: "213" is not among the defined enum values

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "garmin.py", line 389, in <module>
  File "garmin.py", line 379, in main
  File "garmin.py", line 218, in import_data
  File "fit_data.py", line 54, in process_files
  File "fit_file_processor.py", line 83, in write_file
  File "fit_file_processor.py", line 68, in __write_message_types
  File "fit_file_processor.py", line 57, in __write_message_type
  File "fit_file_processor.py", line 46, in __write_generic
  File "fit_file_processor.py", line 478, in _write_device_info_entry
...
sqlalchemy.exc.StatementError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(builtins.LookupError) "213" is not among the defined enum values
[SQL: UPDATE devices SET serial_number=?, timestamp=?, manufacturer=? WHERE devices.serial_number = ?]
[parameters: [{'manufacturer': 213, 'timestamp': datetime.datetime(2017, 5, 14, 15, 20, 29), 'serial_number': '3927983834000008', 'devices_serial_number': 3927983834000008}]]
[11988] Failed to execute script garmin
tcgoetz commented 4 years ago

You have a device that includes a manufacturer id (213) that I haven't seen before, in docs or FIT files. It could be for any device: watch, sensor, etc. Do you know which device it is? Can you share the FIT file that failed with me so I can debug it?

tcgoetz commented 4 years ago

Here's a link to the existing manufacture definitions: https://github.com/tcgoetz/Fit/blob/6d104df61540931a1f5e88f6f0684cc15c8199a2/field_enums.py#L174

AYellowDart commented 4 years ago

I'd be happy to share the file if I could find which one has new manufacturer ID. Any suggestions on how to narrow it down? I don't have any guess as what could be the errant device.

tcgoetz commented 4 years ago

what do the last couple of lines in garmin.log say?

AYellowDart commented 4 years ago

The last 20 or so lines are all similar to this but with different FIT files.

ERROR:root:Exception in _write_running_entry from /Users/wheath/HealthData/FitFiles/Activities/1654927128.fit: 'Cadence' object has no attribute 'to_f' Here's the FIT file in TXT so I could upload it The JSONs of those files all appear to have null as the manufacturer

1654927128.txt

AYellowDart commented 4 years ago

Looks like all these activities were tracked on my "vívoactive HR" by Garmin

tcgoetz commented 4 years ago

1654927128 parses fine for me and I don't see any manufacturer ids other than garmin.

Can you post the log file? What code version are you running?

tcgoetz commented 4 years ago

ERROR:root:Exception in _write_running_entry from /Users/wheath/HealthData/FitFiles/Activities/1654927128.fit: 'Cadence' object has no attribute 'to_f'

commit c0801bd fixes this issue

EmptySoft commented 4 years ago

Hi.!

i have the same issue with other manufacturer code

(builtins.LookupError) "164" is not among the defined enum values [SQL: UPDATE devices SET serial_number=?, timestamp=?, manufacturer=?, product=? WHERE devices.serial_number = ?] [parameters: [{'product': 'UnknownProduct_0', 'manufacturer': 164, 'serial_number': '3923282663000003', 'timestamp': datetime.datetime(2017, 2, 27, 16, 19, 47), 'devices_serial_number': 3923282663000003}]]

witch info do you need for debugging?

tcgoetz commented 4 years ago

Hi.!

i have the same issue with other manufacturer code

(builtins.LookupError) "164" is not among the defined enum values [SQL: UPDATE devices SET serial_number=?, timestamp=?, manufacturer=?, product=? WHERE devices.serial_number = ?] [parameters: [{'product': 'UnknownProduct_0', 'manufacturer': 164, 'serial_number': '3923282663000003', 'timestamp': datetime.datetime(2017, 2, 27, 16, 19, 47), 'devices_serial_number': 3923282663000003}]]

witch info do you need for debugging?

Hi can you open a new issue for this? Can you share file that produced this error? Also what device is this?