Handling of strange .fit files with no body length

jonasinn commented 1 year ago

Hi,

I am currently working on a bit of a toolbox utilizing your library and it is super. The goal for my toolbox is to get all my historical data in to a format where I can work with it, so currently a sql database, but lets see how that goes...

What I ran in to with quite a few files is that they seem to have strange headers with 0 data length and 0 crc, the one I had a bit of a deep dive into was from my old Garmin 1000.

Maybe it is expected behaviour to have the library error out in this case but I forked the library and bodged a bit around. Inside the broken file the data seems OK so I would be keen to hear your opinion if some sort of feature to handle this could be added or if I should except these errors and try to fix them outside the library.

Some things I noticed:

The one file I tested had only the CRC missing, so the EOF error was triggered in at the end of the file where it tried to read 2 bytes but got 0.
The header has 0x00 for all 4 body size bytes and both crc bytes

Cheers, Jonas

fundthmcalculus commented 10 months ago

I've done work on the older fit library to handle data recovery from headunit crashes. I'd say propose your changes, having support for handling partially corrupted files is definitely a useful feature. It's a significant part of what drives people to these tools.

charlieforward9 commented 5 months ago

@jonasinn thanks for the write up.

I am also having an issue with my headers which I wrote up in #30. Specifically, this throws:

# check header size
if header_size < len(chunk) or header_magic != b'.FIT':
    raise FitHeaderError(f'not a FIT file @ {self._chunk_offset}')

Is this related? Does your fork address this?

polyvertex / fitdecode

Handling of strange .fit files with no body length #27