janestreet / memtrace

Streaming client for OCaml's Memprof
MIT License
65 stars 13 forks source link

Correctly clamp `begin_char` and `end_char` when encoding locations #23

Closed lukemaurer closed 1 month ago

lukemaurer commented 1 month ago

The begin_char and end_char fields in Memtrace's trace format are 8 and 10 bits in size, respectively, but they were only being clamped to 12 bits (0xfff). This is particularly bad for the end_char field, as it causes the too-large value to get OR-ed with the filename_code field, which is an index into an MTF cache, so a single error will desync the reader state from the writer, causing a fatal error at some point in the future (fun!).