Code is upcasting quantization table value from u16 to i32 via an i16, which means that 32768+ gets incorrectly promoted to a negative i32. Verified that the C++ version uses a zero-extending upcast from u16. Originally the lepton version also used a zero-extending upcast but at some point I optimized it not realizing the range difference.
This is a somewhat hypothetical difference since 8 bit JPEGs (which are the only ones we allow) aren't allowed to have 16 bit quantization tables by the JPEG standard, although the C++ lepton doesn't actually enforce this, so in theory there could be a bad JPEG that gets compressed this way.
However, now that there are better testcases for these extremes, we can catch errors here.
Code is upcasting quantization table value from u16 to i32 via an i16, which means that 32768+ gets incorrectly promoted to a negative i32. Verified that the C++ version uses a zero-extending upcast from u16. Originally the lepton version also used a zero-extending upcast but at some point I optimized it not realizing the range difference.
This is a somewhat hypothetical difference since 8 bit JPEGs (which are the only ones we allow) aren't allowed to have 16 bit quantization tables by the JPEG standard, although the C++ lepton doesn't actually enforce this, so in theory there could be a bad JPEG that gets compressed this way.
However, now that there are better testcases for these extremes, we can catch errors here.