Endianess issue?

[DanAvni]

Hi

when looking on your library I noticed that for CRC16 (ARC in your standards enumerator), the library returns {0x3D,0xBB} when inface it should be reversed {0xBB,0x3D}. look at http://www.lammertbies.nl/comm/info/crc-calculation.html and other online calculators on the web. They all produce 0xBB3D and yours is the only one producing the wrong endianess

[brandondahler]

I'm afraid you are misunderstanding byte endianess and the way that hexadecimal values are expressed.

When we write the value 0xBB3D, we are expressing them in big-endian as that is how we naturally express numbers in everyday life -- 0xBB3D == 47,933 dec, just like how 0x003D == 61 dec. However, most modern-day, laptop/desktop processors work in little-endian. What this means is that when you read the value 0xBB3D as individual bytes that the computer stores, you'll read it as { 0x3D, 0xBB }.

To put it programmatically, run the following code:

... var crc = new System.Data.HashFunction.CRCStandards.ARC(); var bytes = crc.ComputeHash(System.Text.Encoding.UTF8.GetBytes("123456789")); var shortValue = BitConverter.ToUInt16(bytes);

// Should print BB3D Console.WriteLine(shortValue.ToString("X")); ...

Let me know if you have any other questions or concerns.

[DanAvni]

Hi Brandon, I apologize for not explaining myself correctly (probably a moment of stupidity towards the weekend :-) ). What endianess does the library produce? is it always little? big? or does it vary based on platform? In order for me to compare the CRC the library produces to a CRC provided in a byte stream that has been coded using big endian I need to know the order to compare the bytes.

[brandondahler]

The CRC hash functions are somewhat unique because they are bit-wise hash functions instead of byte-wise hash functions. The bit-endianness of the bytes will be whatever the endianness of the machine is, but that shouldn't matter because the only thing that should matter is the order of the bits.

Since they are bit-wise, the standards specify the endianness that input bytes should be processed and the endianness of the outputted bytes. This is controlled by the reflectIn and reflectOut fields in https://github.com/brandondahler/Data.HashFunction/blob/master/CRC/Standards.json . Since ARC specifies reflectIn and reflectOut as both true, it should process and produce a bitstream that is big-endian.

The good news for you is that this is already handled and you'll find that the bitstream produced by any other endianness machine will produce the same bitstream as you will receive if you do the following:

...
var crc = new System.Data.HashFunction.CRCStandards.ARC();
var bytes = crc.ComputeHash(System.Text.Encoding.UTF8.GetBytes("123456789"));
var bitArray = new BitArray(bytes);

// Should write 1011101100111101
foreach (var bit in bitArray)
    Console.Write((bit ? 1 : 0));

Console.WriteLine();
...

The only thing you might need to be concerned with is the bit-endianness of the bytes when you transmit bytes between the systems -- either the sender needs to convert its bytes from its native endianness to the receivers endianness, or the reciever will need to convert the recieved bytes from the other's endianness its endianness.

Reguardless of the endianness of any of the machines, casting the bytes to a short on the producing machine and printing them out should produce the decimal value 47,933.