On big-endian processors, the CalculateSum() func in core/utils/checksum.h will return an incorrect value when given a buffer with an odd number of bytes. This is because the last byte of the buffer is simply being promoted from 8 to 16 bits:
// Add remaining 8-bit to the one's complement sum
if (odd) {
sum64 += *reinterpret_cast<const uint8_t *>(buf16);
}
This works on LE, but on BE, the last byte should be left-shifted by 8 bits. (Recall, per the RFC, that the last odd byte should be treated as if it's the high-order byte of a 16-bit value in which the low-order byte is zero.)
For example, consider a five-byte buffer containing:
On big-endian processors, the
CalculateSum()
func in core/utils/checksum.h will return an incorrect value when given a buffer with an odd number of bytes. This is because the last byte of the buffer is simply being promoted from 8 to 16 bits:This works on LE, but on BE, the last byte should be left-shifted by 8 bits. (Recall, per the RFC, that the last odd byte should be treated as if it's the high-order byte of a 16-bit value in which the low-order byte is zero.)
For example, consider a five-byte buffer containing:
On BE, the expected adds should be:
but will actually be:
On LE, the adds should be (and will be):
Note that the LE result when byte-swapped equals the correct result from the BE calc, as expected.
This should give the correct result on both BE and LE:
I can provide a test program to simulate this fix on an LE processor.