foxglove / mcap

MCAP is a modular, performant, and serialization-agnostic container file format, useful for pub/sub and robotics applications.
https://mcap.dev
MIT License
523 stars 96 forks source link

go: indexed message iterator: guard against bad offsets in file #1234

Closed james-rms closed 1 month ago

james-rms commented 1 month ago

Changelog

Docs

None.

Description

There are files being uploaded to Foxglove that have invalid offsets inside them - these cause errors in io.Seek, which can't be told apart from true IO errors at the moment. This means if a read fails, we can't tell if it's because of a corrupt file or flaky I/O. This PR lets us disambiguate by returning a specific error from the library.

BeforeAfter
james-rms commented 1 month ago

I think UnexpectedEOF would be confusing here. We only return this error after we've already seeked to the end of the file and checked for end magic, so we know where the file ends and that it isn't truncated. This error indicates corruption, by the writer or through bit flips, rather than the file being truncated.