polyvertex / fitdecode

A FIT file parsing and decoding library written in Python3
MIT License
157 stars 34 forks source link

Assertion Error during Read #8

Closed dalexnagy closed 4 years ago

dalexnagy commented 4 years ago

Thanks for putting this together!

I am using it in a Python program to get information from my Wahoo Elemnt bike computer. The Elemnt uploads the FIT file to DropBox where I can then access it on my computer (Ubuntu Linux).

For one file, my processing fails at the 'for frame in fit_frame' statement with an 'AssertionError' (see messages pasted below code snippet).

I can get around this error with a 'try/except' block (shown in the code snippet with comments indicators) but that entire FIT file is skipped. I'd like to be able to skip that frame and go on to the next one. My code only processes frames that I want to see.

1) Is there a way for me to handle this 'AssertionError' and continue processing the rest of this file as most of the data I'm looking for is in a frame late in the file (ie, written when I mark the cycling activity as 'done')? 2) Any other suggestions on how to handle this error will be greatly appreciated!

If more code can help, I can post the entire program - this one is only a test so I could see what the Wahoo was uploading.

Thanks! Dave

CODE SNIPPET: for fit_file in fit_files: frame_no = 0 fit_file = src_dir+"/"+fit_file with fitdecode.FitReader(fit_file,keep_raw_chunks=True) as fit_frame:

try:

        for frame in fit_frame:
            frame_no+=1
            if isinstance(frame, fitdecode.FitDataMessage):
                try:
                    name_idx = frame_names.index(frame.name)
                except ValueError:
                    name_idx = -1

                if name_idx > -1:
                    fields = frame.fields
                    for f in fields:
                        out_file.write(str(frame_no)+"#["+frame_names[name_idx]+"] Name:"+f.name+
                              ", def_num:"+str(f.def_num)+
                              ", Type:"+f.type.name+
                              ", Value:"+str(f.value)+
                              ", Units:"+str(f.units)+"\n")
                    out_file.write("\n")

except AssertionError:

print("Assertion failure! Last frame was",frame_no)

ERROR MESSAGES: Traceback (most recent call last): File "/home/dave/Python/FIT/test1.py", line 26, in for frame in fit_frame: File "/home/dave/FITFiles/lib/python3.6/site-packages/fitdecode/reader.py", line 193, in iter yield from self._read_next() File "/home/dave/FITFiles/lib/python3.6/site-packages/fitdecode/reader.py", line 304, in _read_next assert self._chunk_size <= self._body_bytes_left AssertionError

dxkaufman commented 4 years ago

Hi there!

I am having a similar problem, but I get a KeyError and there's no traceback going into the fitdecode code:

Exception has occurred: KeyError 'field "units" (idx #0) not found in message "field_description"' File "/Users/dylankaufman/Documents/Projects/GPS/main.py", line 113, in for frame in fit:

I even moved the call to the units attribute to a separate function hoping that would keep it from complaining about that. Here's the code:

        with fitdecode.FitReader(data) as fit:
            for frame in fit:
                if isinstance(frame, fitdecode.FitHeader):
                    print('header')
                elif isinstance(frame, fitdecode.FitDefinitionMessage):
                    print('definition')
                    for field_def in frame.field_defs:
                        print(f'field_def: {field_def.name}')
                elif isinstance(frame, fitdecode.FitCRC):
                    print('crc')
                elif isinstance(frame, fitdecode.FitDataMessage):
                    # Here, frame is a FitDataMessage object.
                    # A FitDataMessage object contains decoded values that
                    # are directly usable in your script logic.
                    process_fields(frame, fit_catalog)
                else:
                    print('unknown frame type')

I am also putting a try on it, and will log which file(s) have this error so I can figure out which they are and try to download them in gpx or tcx instead of fit...

Would appreciate any other suggestions you might have.

dalexnagy commented 4 years ago

I found that my assertion errors were due to an incomplete FIT file and this occurred because of an upload failure. I fixed it by uploading the FIT file again - I use a Wahoo Elemnt and I can do a manual upload to Dropbox. It is rare for this to happen but it does and I now check the status of the upload after my ride - If I don't see the message that a ride was uploaded to 3 sites (Dropbox being one of these), I can force it again. Best of luck!

dxkaufman commented 4 years ago

I haven't figured out whether it is an incomplete file, or what. I was able to identify the dates from the 4 that were causing errors and download them individually from Strava as gpx files, and my gpx processing code didn't object to anything. Of course, so far all I'm doing is extracting all the node names and data types, so we'll see what happens when I try to convert them all into a bunch of data tables.

polyvertex commented 4 years ago

For the lack of better debugging support, this kind of error usually happen because of corrupted or incomplete files, which seems to be quite frequent.

Closing this due to inactivity.