Error parsing TLV with variable length data (array, choice)

[wpyoga]

When parsing a TLV with variable-length data (choice inside array), these methods don't work:

• length: "$parent.length"
• length: "length"
• length: function(vars) {return vars.length;}

If we try to access "$parent.length" (supposed to be introduced by #166), then ts-node will throw an error:

TypeError: Cannot read properties of undefined (reading 'length')

If we use "length", then parsing is abruptly stopped, although no error seems to be thrown.

If we use function(vars) as mentioned in #126, it also doesn't work, and we get the same result as if we are using "length".

If we specify a literal length like length: 3, which is known in advance (see 0xff type below) then it works. It just doesn't work if the length is read from the data itself.

Using .useContextVars() doesn't help at all.

Example, with multiple commented lines that can be uncommented in order to see the issue in action:

import { Parser } from 'binary-parser';

// a1012a a2027ab8 a30460713d44 ff03414243 a401ff
const buf = Buffer.from('a1012aa2027ab8a30460713d44ff03414243a401ff', 'hex');

const parser = new Parser().array('data', {
  readUntil: 'eof',
  type: new Parser()
    // this doesn't help:
    // .useContextVars()
    .uint8('type')
    .uint8('length')
    .choice('value', {
      tag: 'type',
      choices: {
        0xa1: new Parser().uint8('_a1'),
        0xa2: new Parser().uint16be('_a2'),

        // this works:
        0xa3: new Parser().uint32be('_a3'),
        // this doesn't work:
        // 0xa3: new Parser().buffer('_a3', { length: 'length' }),
        // this doesn't work:
        // 0xa3: new Parser().buffer('_a3', {
        //   length: function (vars) {
        //     return vars.length;
        //   },
        // }),
        // this produces error:
        // TypeError: Cannot read properties of undefined (reading 'length')
        // 0xa3: new Parser().buffer('_a3', { length: '$parent.length' }),

        0xa4: new Parser().uint8('_a4'),

        // uncomment this to skip over defaultChoice and parse the a4 type
        // 0xff: new Parser().buffer('_ff', { length: 3 }),
      },

      // this produces error
      // TypeError: Cannot read properties of undefined (reading 'length')
      // defaultChoice: new Parser().buffer('other', { length: '$parent.length' }),
      // this doesn't work
      defaultChoice: new Parser().buffer('other', { length: 'length' }),
      // same, doesn't work
      // defaultChoice: new Parser().buffer('other', {
      //   length: function (vars) {
      //     return vars.length;
      //   },
      // }),
    }),
});

console.dir(parser.parse(buf), { depth: null });

Am I doing something wrong here...?

[keichi]

Sorry for the unclarity, you need to call useContextVars() on the top-level parser. If you change

const parser = new Parser().array('data', {

to

const parser = new Parser().useContextVars().array('data', {

This works:

const Parser = require('binary-parser').Parser;

const buf = Buffer.from('a1012aa2027ab8a30460713d44ff03414243a401ff', 'hex');

const parser = new Parser().useContextVars().array('data', {
  readUntil: 'eof',
  type: new Parser()
    .uint8('type')
    .uint8('length')
    .choice('value', {
      tag: 'type',
      choices: {
        0xa1: new Parser().uint8('_a1'),
        0xa2: new Parser().uint16be('_a2'),
        0xa3: new Parser().buffer('_a3', { length: '$parent.length' }),
        0xa4: new Parser().uint8('_a4'),
      },
      defaultChoice: new Parser().buffer('other', { length: '$parent.length' }),
    }),
});

console.dir(parser.parse(buf), { depth: null });

Output:

{
  data: [
    { type: 161, length: 1, value: { _a1: 42 } },
    { type: 162, length: 2, value: { _a2: 31416 } },
    {
      type: 163,
      length: 4,
      value: { _a3: Buffer(4) [Uint8Array] [ 96, 113, 61, 68 ] }
    },
    {
      type: 255,
      length: 3,
      value: { other: Buffer(3) [Uint8Array] [ 65, 66, 67 ] }
    },
    { type: 164, length: 1, value: { _a4: 255 } }
  ]
}

[wpyoga]

I see... your solution works. I'm curious though, does this mean useContextVars() have to declared at the very top level parser, and at that level only?

[keichi]

Yes, you have to declare it at the top level. If you call it at the lower levels it's simply ignored. This is required because the top-level parser has to know if the context vars is enabled as it sets$root.

[keichi]

Closing since your initial problem seems resolved. Feel free to reopen if needed.

[wpyoga]

I see, thank you for the explanation.