Problem with decoding custom ASN1 messages

Hello!

Not necessarily a bug more of an issue when receiving messages from an application with specific rules

https://github.com/lapo-luchini/asn1js/blob/6fa69ff58a421794078f4f07358d3da17526808c/asn1.js#L442-L450

There are cases when an application specifically considers that the client should parse a BitString, however the recurse variable defined in Line 332:

avoidRecurse = false

is triggered. This cause the JS client to be unable to parse the correct value contained within it. It would be very nice to be able to force this variable to force under certain circumstances.

Example hex:

30820c92020100031100890e6dada073475c839b216850c19f48030500424f4f4b0206018ec40d8070010100020100303913074254432d555344031100f5620ed9e64f4b34a1775f4d360f42a013034254430311009a399da1ba214649b80eb81515a38eb71303555344130653435259505403820c200030820c1b3081fe303d130836383738352e31343031302f05000500050002013105000500050005000500050005000500130a342e3030303030303030130836383738352e3134303d130836383737352e34303031302f05000500050002013105000500050005000500050005000500130a352e3030303030303030130836383737352e3430303e130836383736362e36393032303005000500050002013105000500050005000500050005000500130b33312e3030303030303030130836383736362e3639303e130836383735332e37303032303005000500050002013105000500050005000500050005000500130b36302e3030303030303030130836383735332e373030820b16303d130836383836372e33393031302f05000500050002013205000500050005000500050005000500130a312e3030303030303030130836383836372e3339303d130836383838312e36353031302f05000500050002013205000500050005000500050005000500130a342e3030303030303030130836383838312e3635303d130836383839312e38383031302f05000500050002013205000500050005000500050005000500130a352e3030303030303030130836383839312e3838303e130836383930312e30313032303005000500050002013205000500050005000500050005000500130b34302e3030303030303030130836383930312e3031303e130836383931322e38323032303005000500050002013205000500050005000500050005000500130b35302e3030303030303030130836383931322e3832303d130836383933362e36323031302f05000500050002013205000500050005000500050005000500130a302e3532363632373331130836383933362e3632303d130836383933372e31323031302f05000500050002013205000500050005000500050005000500130a342e3030303030303030130836383933372e3132303d130836383933382e35323031302f05000500050002013205000500050005000500050005000500130a312e3239313430383531130836383933382e3532303d130836383934312e31323031302f05000500050002013205000500050005000500050005000500130a302e3134353230303030130836383934312e3132303d130836383934322e36333031302f05000500050002013205000500050005000500050005000500130a322e3137373436353735130836383934322e3633303d130836383934382e39333031302f05000500050002013205000500050005000500050005000500130a302e3333373530303030130836383934382e3933303d130836383935312e32333031302f05000500050002013205000500050005000500050005000500130a302e3032353532313533130836383935312e3233303d130836383935342e30333031302f05000500050002013205000500050005000500050005000500130a302e3330373037353038130836383935342e3033303d130836383935342e31333031302f05000500050002013205000500050005000500050005000500130a302e3333373530303030130836383935342e3133303d130836383935362e34343031302f05000500050002013205000500050005000500050005000500130a302e3839303030303030130836383935362e3434303d130836383935382e33343031302f05000500050002013205000500050005000500050005000500130a322e3030303434323439130836383935382e3334303d130836383935382e35343031302f05000500050002013205000500050005000500050005000500130a302e3035353431303030130836383935382e3534303d130836383935382e38343031302f05000500050002013205000500050005000500050005000500130a302e3733323536333532130836383935382e3834303d130836383935392e33343031302f05000500050002013205000500050005000500050005000500130a302e3333373530303030130836383935392e3334303d130836383936332e31343031302f05000500050002013205000500050005000500050005000500130a302e3031303030303030130836383936332e3134303d130836383936342e31303031302f05000500050002013205000500050005000500050005000500130a372e3535343437303030130836383936342e3130303d130836383936372e38353031302f05000500050002013205000500050005000500050005000500130a302e3933353136313333130836383936372e3835303d130836383936382e34353031302f05000500050002013205000500050005000500050005000500130a302e3031393634343933130836383936382e3435303d130836383936392e37353031302f05000500050002013205000500050005000500050005000500130a302e3333373530303030130836383936392e3735303d130836383937322e36353031302f05000500050002013205000500050005000500050005000500130a362e3736353332393439130836383937322e3635303d130836383937362e31353031302f05000500050002013205000500050005000500050005000500130a302e3239363238313239130836383937362e3135303d130836383937362e37353031302f05000500050002013205000500050005000500050005000500130a302e3031393634313835130836383937362e3735303d130836383938302e38363031302f05000500050002013205000500050005000500050005000500130a302e3232313434313732130836383938302e3836303d130836383938302e39363031302f05000500050002013205000500050005000500050005000500130a302e3333373530303030130836383938302e3936303d130836383938362e31363031302f05000500050002013205000500050005000500050005000500130a302e3333373530303030130836383938362e3136303d130836383938382e31363031302f05000500050002013205000500050005000500050005000500130a302e3030313138383238130836383938382e3136303d130836383938392e30363031302f05000500050002013205000500050005000500050005000500130a302e3231393030303030130836383938392e3036303d130836383939322e31363031302f05000500050002013205000500050005000500050005000500130a332e3030303030303030130836383939322e3136303d130836393030342e32373031302f05000500050002013205000500050005000500050005000500130a302e3334303038323534130836393030342e3237303d130836393030372e38383031302f05000500050002013205000500050005000500050005000500130a302e3030323237373137130836393030372e3838303d130836393133392e31383031302f05000500050002013205000500050005000500050005000500130a312e3737353131383333130836393133392e3138303d130836393138322e30353031302f05000500050002013205000500050005000500050005000500130a352e3334313938323030130836393138322e3035303d130836393232312e34363031302f05000500050002013205000500050005000500050005000500130a302e3030313038333436130836393232312e3436303d130836393330352e34303031302f05000500050002013205000500050005000500050005000500130a332e3534323037373738130836393330352e3430303d130836393330352e34323031302f05000500050002013205000500050005000500050005000500130a302e3031333838383838130836393330352e3432303d130836393331312e37323031302f05000500050002013205000500050005000500050005000500130a302e3034343139303535130836393331312e3732303e130836393332322e38333032303005000500050002013205000500050005000500050005000500130b31302e3037373634393731130836393332322e3833303d130836393334342e32393031302f05000500050002013205000500050005000500050005000500130a302e3031373531323839130836393334342e3239303d130836393334342e34353031302f05000500050002013205000500050005000500050005000500130a302e3333313930393036130836393334342e3435303d130836393335352e34363031302f05000500050002013205000500050005000500050005000500130a302e3337373839363432130836393335352e3436

When decoding this, you can find the following data:

telegram-cloud-photo-size-1-5525983349592664006-y

In this case the BitString should have just remained a BitString of length 128 but it gets decoded into an unknown type 9 with a length of 14 bytes.

I checked this point using another lib (https://www.npmjs.com/package/asn1js) using this method:

fromBER(buffer).result.valueBlock.value[1] (to pick up a only problem string)

where buffer is an hex code example above (30820c92020100031100890e6dada073475c839b216850c1...)

and saw this object:

{
  "blockName": "BIT STRING",
  "blockLength": 19,
  "error": "",
  "warnings": [],
  "valueBeforeDecode": "031100890e6dada073475c839b216850c19f48",
  "idBlock": {
    "blockName": "identificationBlock",
    "blockLength": 1,
    "error": "",
    "warnings": [],
    "valueBeforeDecode": "",
    "isHexOnly": false,
    "valueHex": "",
    "tagClass": 1,
    "tagNumber": 3,
    "isConstructed": false
  },
  "lenBlock": {
    "blockName": "lengthBlock",
    "blockLength": 1,
    "error": "",
    "warnings": [],
    "valueBeforeDecode": "",
    "isIndefiniteForm": false,
    "longFormUsed": false,
    "length": 17
  },
  "valueBlock": {
    "blockName": "BitStringValueBlock",
    "blockLength": 17,
    "error": "",
    "warnings": [],
    "valueBeforeDecode": "",
    "isIndefiniteForm": false,
    "value": [
      {
        "blockName": "PRIMITIVE",
        "blockLength": 16,
        "error": "",
        "warnings": [],
        "valueBeforeDecode": "890e6dada073475c839b216850c19f48",
        "idBlock": {
          "blockName": "identificationBlock",
          "blockLength": 1,
          "error": "",
          "warnings": [],
          "valueBeforeDecode": "",
          "isHexOnly": false,
          "valueHex": "",
          "tagClass": 3,
          "tagNumber": 9,
          "isConstructed": false
        },
        "lenBlock": {
          "blockName": "lengthBlock",
          "blockLength": 1,
          "error": "",
          "warnings": [],
          "valueBeforeDecode": "",
          "isIndefiniteForm": false,
          "longFormUsed": false,
          "length": 14
        },
        "valueBlock": {
          "blockName": "PrimitiveValueBlock",
          "blockLength": 14,
          "error": "",
          "warnings": [],
          "valueBeforeDecode": "",
          "isHexOnly": true,
          "valueHex": "6dada073475c839b216850c19f48"
        },
        "name": "",
        "optional": false
      }
    ],
    "isHexOnly": false,
    "valueHex": "890e6dada073475c839b216850c19f48",
    "unusedBits": 0,
    "isConstructed": false
  },
  "name": "",
  "optional": false
}

the most interesting thing is a field:

valueBlock.value.valueBlock.valueHex: "6dada073475c839b216850c19f48"

which is not valid for us

but, we also have another value from this object

"valueHex": "890e6dada073475c839b216850c19f48"

which is exactly what we need so, I would like to understand why this happened. there is a feeling that the line "890e6dada073475c839b216850c19f48 "has undergone additional processing

and of course I wouldn't want to use this library, because I need process each value and also this library works twice as slow

The problem is that unfortunately many "interesting" ASN.1 structures hide content inside either BIT STRINGs or OCTET STRINGs, and since asn1js has a "best effort" decoding approach it tries decoding the content of any of those and if it "just happens" to be valid ASN.1 it shows the content as "embedded" (instead of "constructed", which means it was done with proper ASN.1 structures). As you can notice (while hovering the BIT STRING, not the content) is is made so: 03 11 00 89 0E 6D AD A0 73 47 5C 83 9B 21 68 50 C1 9F 48 Where 03 mean it's a BIT STRING, 11 means it is 17 bytes long, 00 is the number of unused bits (0 means the last byte is fully used), then the content which in this case is raw 89 0E 6D AD A0 73 47 5C 83 9B 21 68 50 C1 9F 48. Problem is, 89 means custom tag [9] and 0E means 14 byte long, which actually match the rest of the content. This is most probably a false positive, but it is unavoidable (while keeping the approach of decoding as much as possible for "hidden" internal structures, which are not rare).

You have the very same behavior that that other library you cite which I have no experience with, but they do basically the same I do in returnig you both the direct hex encoding of the current node, and a way to access inner structures with .value.valueBlock.

there is a feeling that the line "890e6dada073475c839b216850c19f48 "has undergone additional processing

Actualyl this is the raw content of the BIT STRING, it's the "6dada073475c839b216850c19f48" that actually has undergone additional processing, in trying to decode as inner value.

If you are using asn1js and never want to go inside "encapsulated" structures, you can do that by checking for that explicitly, there is currently no way to disable recursion altogether.

Would you like a checkbox in the GUI to avoid any recursion? (this is not a great approach when you have big ASN.1 structures with both BIT STRINGs that use recursion and ones that should not, though)

Or maybe, when using as a library, I could make it more easy to understand when a node is an actual "encapsulation" (which might be a false positive).

Actually your very example contains both an example you say is a "false" content, and a node that, from its complexity, I'd say it's DEFINITELY a real encapsulated content:

I could make it more easy to understand when a node is an actual "encapsulation" (which might be a false positive).

Actually it's not very difficult (while not very documented), you could do it like this:

if (this.sub !== null) { // has inner structures
    if (this.tag.tagConstructed)
        // this is a constructed content (for sure)
    else
        // this is a (supposedly) encapsulated content
}

There are cases when an application specifically considers that the client should parse a BitString, however the recurse variable defined in Line 332:

BTW I'm not sure I understood this correctly, as the rest of you message seems (to me) to be about a structure which was erroneously decoded, not a structure that hasn't been decoded but should have been.

Can you clarify?

lapo-luchini / asn1js

Problem with decoding custom ASN1 messages #85