NetSys / bess

BESS: Berkeley Extensible Software Switch
Other
313 stars 156 forks source link

Incorrect checksum calc for odd-sized buffers on big-endian #1044

Open ccpearson opened 2 years ago

ccpearson commented 2 years ago

On big-endian processors, the CalculateSum() func in core/utils/checksum.h will return an incorrect value when given a buffer with an odd number of bytes. This is because the last byte of the buffer is simply being promoted from 8 to 16 bits:

  // Add remaining 8-bit to the one's complement sum
  if (odd) {
    sum64 += *reinterpret_cast<const uint8_t *>(buf16);
  }

This works on LE, but on BE, the last byte should be left-shifted by 8 bits. (Recall, per the RFC, that the last odd byte should be treated as if it's the high-order byte of a 16-bit value in which the low-order byte is zero.)

For example, consider a five-byte buffer containing:

aabb ccdd ee

On BE, the expected adds should be:

     aabb
   + ccdd
   + ee00
    ~~~~~
    26598 = 659a (folded to 16 bits)

but will actually be:

     aabb
   + ccdd
   + 00ee
    ~~~~~
    17886 = 7887 (folded)

On LE, the adds should be (and will be):

     bbaa
   + ddcc
   + 00ee
    ~~~~~
    19a64 = 9a65 (folded)

Note that the LE result when byte-swapped equals the correct result from the BE calc, as expected.

This should give the correct result on both BE and LE:

   sum64 += be16_t(uint16_t(*reinterpret_cast<const uint8_t *>(buf16)) << 8).raw_value();

I can provide a test program to simulate this fix on an LE processor.