search

psforever / psemu

A C++ rewrite of the Scala PSForever server.
8 stars 5 forks source link

Respecting Endianness When Reading Bits #2

Closed Fate-JH closed 7 years ago

Fate-JH commented 7 years ago

First. :stuck_out_tongue:

Currently, some of the reading and writing functions don't support proper endianness. I can not confirm whether big endianness is supported at all. I already spoke with @tfarley about this but, since we're being responsible developers, making an issue on GitHub is appropriate.

To remind developers how it works according to PlanetSide: Endianness is a method by which the byte representation of a number expands as that number increases in value. Conventional base-10 integer math is an example of big endian growth. As the number gets higher, more columns to represent the more significant digits are added to the left of the earlier, currently-most significant column. 1 + 9 = 10, where the maximum value for a "ones" column is exceeded and the extra is carried over and displayed in the "tens" column. As bytes, the highest hexadecimal value that can be stored is 255, as 0xFF. If 1 is added, there is overflow; but, with more capacity, it can be represented in the next byte as the big endian 0x0100. Counting continues with 0x0101 and 0x0102.

(Windows Calculator, in Programmer display, Hex entry mode, accepts numbers typed in big endian form. Enter Hex 1-0-0 and switch to decimal.)

Little endianness is the more common representation for numerical PlanetSide packet data and is the opposite of big endian. As a number grows in value, the representation of that number grows to the right. In bytes, where 1 is added to 0xFF again, the little endian representation becomes 0x0001. Counting continues with 0x0101 and 0x0201.

To translate a number between representations, you reverse the order of the bytes around the right-most one. The 3-byte little endian number 0x6C2D70 is read as the big endian number 0x702D6C.

One minor detail is when endianness runs into situations of not having enough bits to fill out a full byte or a full nibble. In big endian form, a 10-bit number has a maximum value of 0x3FF, which is the same value as 0x03FF. Note that the first byte is the FF and the "second byte" is the two bits that become the 3. In little endian form, that same ten bit number (1023) is represented as 0xFF3 but may be encountered in live as 0xFFC. Likewise, 0x3FF might be encountered as part of an 0x7FF and the value starts from the third bit from the left. A better way to understand this is to look at the bit representations: Little endian: 0x1111 1111 11-- Big endian: 0x--11 1111 1111 Dashes represent ignored bits. When reading the odd number of bits, they are interpretted, that is to say, as if part of a normal byte (0x_0000 00_11 == 3). Returning to the previous example above, the 20-bit little endian number 0x6C2D7 is big endian 0x72D6C.

Fate-JH commented 7 years ago

Wait, #2? There aren't any closed issues. I've been gypped.

ghost commented 7 years ago

Which functions suffer from this problem? (out of curiosity, doesn't have to be an exhaustive list)

And how does the Endianness support affect the runtime of the application? Or to rephrase, where are we receiving large Endian numbers from?

Fate-JH commented 7 years ago

At the moment, anything that transmits player coordinates or velocity would suffer from this because the game is looking for a 20-bit little endian representations and 14-bit little endian representations, and I couldn't read that properly last time I checked. So, yeah, that's a huge deal if the server thinks that the LE 0x6C2D7 it receives should be read as BE 0x6C2D7 (or as BE 0x7D2C6, as the case were).

As to big endian numbers, we currently have two uses for them. One use is strings of 128+ characters. We can't just say "we'll never send the client anything above 127" because that's not going to fly - the client can send the server something over 127 characters. The other use is for large inventories where the client and the server are talking about slot positions that need to be represented by more than 127. Lockers, MAX suit inventories, and some vehicles, and whenever something is placed "in your hand"/"on your cursor."