keichi / binary-parser

A blazing-fast declarative parser builder for binary data
MIT License
857 stars 133 forks source link

Difference between bit8 and uint8 #239

Open nick-hunter opened 1 year ago

nick-hunter commented 1 year ago

I'm currently writing an application that creates several hundred different parsers based on structures defined in a database. I have a lookup table that converts the datatype from the DB into a type from binary-parser. I discovered a difference between bit8 and uint8 and I'm wondering if this is expected behavior or not.

const buff = Buffer.from("7B2D", "hex");
const p1 = new Parser().bit1("one").uint8("eight") // { one: 0, eight: 45 }
const p2 = new Parser().bit1("one").bit8("eight") // { one: 0, eight: 246 }

Here is the generated code for both parsers

// p1
var dataView = new DataView(buffer.buffer, buffer.byteOffset, buffer.length);
var offset = 0;
var vars = {};
vars.$parent = null;
vars.$root = vars;
var $tmp0 = 0;
$tmp0 = dataView.getUint8(offset);
offset += 1;
vars.one = $tmp0 >> 7 & 0x1;
vars.eight = dataView.getUint8(offset, false);
offset += 1;
delete vars.$parent;
delete vars.$root;
return vars;

// p2
var dataView = new DataView(buffer.buffer, buffer.byteOffset, buffer.length);
var offset = 0;
var vars = {};
vars.$parent = null;
vars.$root = vars;
var $tmp0 = 0;
$tmp0 = dataView.getUint16(offset);
offset += 2;
vars.one = $tmp0 >> 15 & 0x1;
vars.eight = $tmp0 >> 7 & 0xff;
delete vars.$parent;
delete vars.$root;
return vars;

The code for p1 reads one bit, skips 7 bits, and starts reading at the start of the next byte. Is this expected? I didn't see anything in the docs about this difference. I'm thinking about tracking both the bit and byte position, and rewriting uint8 to bit8 if the bit offset != 0. Just wanted to see if anyone else has run into similar issues.

keichi commented 1 year ago

Yes, this is by design and we should document it. Basically, anything except bit fields needs to be byte-aligned. This is to simplify the parser builder code (otherwise we would need to keep track of bit and byte positions as you mentioned).