Long-From of `Identifier octets`

bobaxix commented 9 months ago

Description

BER encoding defines, that ASN1 tags can be encoded as 1 or more bytes (according to Tag Type value)

https://en.wikipedia.org/wiki/X.690#BER_encoding

It was intentional to use as tag always first byte of encodedBytes?

https://github.com/wstrange/asn1lib/blob/master/lib/src/asn1parser.dart#L29C62 https://github.com/wstrange/asn1lib/blob/master/lib/src/asn1object.dart#L71

To reproduce

import 'dart:typed_data';
import 'package:asn1lib/asn1lib.dart';

// Tag is 0x5F21, what means:
//
// - Tag class: 0x01 (Application)
// - P/C: 0 (Primitive)
// - Tag type: 0x21
final values= Uint8List.fromList(
  [
    0x5F,
    0x21,
    0x81,
    0x01,
    0x7F,
  ],
);

void main() {
  var asn1Parser = ASN1Parser(
    values,
  );

  //Throws:
  //
  //Unhandled exception:
  //Instance of 'ASN1Exception'
  //#0      ASN1Parser._doPrimitive (package:asn1lib/src/asn1parser.dart:136:9)
  //#1      ASN1Parser.nextObject (package:asn1lib/src/asn1parser.dart:56:15)
  //#2      main (file:///C:/Users/marcinb/Desktop/testx/bin/testx.dart:19:20)
  //#3      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:296:19)
  //#4      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:189:12)
  print(asn1Parser.nextObject());
}

wstrange commented 9 months ago

Not 100% sure here. Shot in the dark..

Where are those bytes coming from? It kinda look like the first byte (0x5F) is not part of the asn1 stream?

If I comment out that first byte, and enable relaxed parsing (i.e. tells the parser to parse unknown application types as a plain old ASN1Object). It kinda works:

import 'dart:typed_data';
import 'package:asn1lib/asn1lib.dart';

// Tag is 0x5F21, what means:
//
// - Tag class: 0x01 (Application)
// - P/C: 0 (Primitive)
// - Tag type: 0x21
final values= Uint8List.fromList(
  [
    // 0x5F,
    0x21,
    0x81,
    0x01,
    0x7F,
  ],
);

void main() {
  var asn1Parser = ASN1Parser(
    values,
    relaxedParsing: true,
  );

  //Throws:
  //
  //Unhandled exception:
  //Instance of 'ASN1Exception'
  //#0      ASN1Parser._doPrimitive (package:asn1lib/src/asn1parser.dart:136:9)
  //#1      ASN1Parser.nextObject (package:asn1lib/src/asn1parser.dart:56:15)
  //#2      main (file:///C:/Users/marcinb/Desktop/testx/bin/testx.dart:19:20)
  //#3      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:296:19)
  //#4      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:189:12)
  print(asn1Parser.nextObject());
}

Gives me the output:

ASN1Object(tag=21 valueByteLength=1) startpos=3 bytes=[0x21, 0x81, 0x1, 0x7f]

wstrange commented 9 months ago

Also, are there possibly extra bytes needed in that stream? If I run the debugger on your original bytes, the length is calculated as 35 bytes. That is past the end of that stream.

bobaxix commented 9 months ago

@wstrange

Comments

Sequence is dummy - dont be particular about it.
Look there https://luca.ntop.org/Teaching/Appunti/asn1.html (document from README.md)

Short encoding:

Low-tag-number form. One octet. Bits 8 and 7 specify the class (see Table 2), bit 6 has value "0," indicating that the encoding is primitive, and bits 5-1 give the tag number.

Long encoding:

High-tag-number form. Two or more octets. First octet is as in low-tag-number form, except that bits 5-1 all have value "1." Second and following octets give the tag number, base 128, most significant digit first, with as few digits as possible, and with the bit 8 of each octet except the last set to "1."

I know that "simple types" of ASN.1 contains only this values. In one byte tag we have 5 bits for tag number, so we can encode values from 0 to 31

But from this ASN1 contains also values above this range (example: DURATION (34₁₀, 22₁₆). How you encode it on five bits?

Real life example

I working with tachographs, and there are special cards to store information about company/driver etc. Cards are protected via certification system which use certificate chains. Certificate always starts with application specific sequence tag 7F 21 [len].

In your lib I cannot read sequence because 21 is treat as length, not [len] field.

Post-scriptum

I'm not expert in ASN1 and BER so maybe something is missing in my mind. I also check another lib to ASN1 encoding (https://pub.dev/packages/pointycastle) and there also only one byte is used as tag, so it suggest my error.

wstrange commented 9 months ago

I think it would help to provide a proper ASN1 sequence (not dummy bytes). We can further diagnose the problem.

bobaxix commented 9 months ago

@wstrange especially for you:

final certificate = Uint8List.fromList(
[
    0x7F, 0x21, 0x81, 0xC9, 0x7F, 0x4E, 0x81, 0x82, 0x5F, 0x29, 0x01, 0x00, 0x42, 0x08, 0xFD, 0x45,
    0x43, 0x20, 0x01, 0xFF, 0xFF, 0x01, 0x5F, 0x4C, 0x07, 0xFF, 0x53, 0x4D, 0x52, 0x44, 0x54, 0x0D,
    0x7F, 0x49, 0x4E, 0x06, 0x09, 0x2B, 0x24, 0x03, 0x03, 0x02, 0x08, 0x01, 0x01, 0x07, 0x86, 0x41,
    0x04, 0x08, 0xC0, 0x4E, 0x39, 0x26, 0xC8, 0xDE, 0x85, 0x54, 0x42, 0x40, 0xCD, 0xE4, 0x0D, 0xAB,
    0x70, 0xD2, 0xB4, 0x7E, 0x0F, 0x83, 0x76, 0x25, 0x22, 0xD7, 0xB0, 0xB8, 0x54, 0x3B, 0x9B, 0x29,
    0xDC, 0x80, 0xE5, 0xC6, 0x7B, 0x82, 0xA6, 0x2D, 0x55, 0xE3, 0x48, 0x3A, 0xB4, 0xB0, 0x0A, 0x24,
    0xC2, 0xA2, 0x56, 0x6C, 0x37, 0x86, 0x79, 0x7A, 0x1A, 0x05, 0x28, 0x22, 0xAB, 0x4B, 0xF1, 0xF2,
    0x92, 0x5F, 0x20, 0x08, 0xFD, 0x45, 0x43, 0x20, 0x01, 0xFF, 0xFF, 0x01, 0x5F, 0x25, 0x04, 0x5B,
    0x21, 0xB0, 0x00, 0x5F, 0x24, 0x04, 0x9B, 0x8F, 0xAE, 0x80, 0x5F, 0x37, 0x40, 0x65, 0xC6, 0x2A,
    0xC1, 0x3D, 0xED, 0x14, 0x7F, 0xA8, 0xD1, 0xD1, 0x1A, 0x8F, 0x5B, 0xF2, 0xCF, 0x9E, 0x95, 0xDB,
    0x1B, 0x43, 0xD2, 0x53, 0xB4, 0x8B, 0x61, 0x5B, 0x2F, 0xE7, 0x0B, 0x3F, 0xD8, 0x2A, 0xA8, 0xD3,
    0x3D, 0x27, 0xF0, 0xF4, 0xD7, 0x36, 0x7C, 0x04, 0x90, 0x3B, 0xBB, 0xE6, 0x37, 0x5B, 0x64, 0x3A,
    0x19, 0xC5, 0xB8, 0x3D, 0x19, 0xFC, 0x74, 0x85, 0xDB, 0x47, 0x6C, 0x70, 0x67,
  ],
);

You can download it from here ERCA Gen2 (1) Root Certificate.bin

wstrange commented 9 months ago

Is that an x509 certificate? Have you looked at https://pub.dev/packages/x509 for parsing it?

How sure are you that all of those bytes are ASN1 encoded ?

wstrange commented 9 months ago

FWIW, openssl does not think that PEM file is a certificate. I'm really lost on what format that root cert is supposed to be.

➜ cert openssl x509 -in t.pem -text Could not find certificate from t.pem

Is there another utility or language library that correctly parses that PEM file?

bobaxix commented 9 months ago

Why do you assume that is X509 certificate ?

Please dont focus on data type, using ASN1 you can encode any data. One and only question is that tag in TLV can be more that 1 byte.

Here, look for CSM_134 „this” certificate content is described.

wstrange commented 9 months ago

Do you have an example of another library or CLI utility that can parse that data? i.e. so we can compare

wstrange commented 9 months ago

None of the previous applications of this library need more than one byte for the tag. So if this format needs > 1 it is going to be a fairly large change.

I'm not sure when I'd get to this. If you'd consider a PR I would take a look

bobaxix commented 9 months ago

Yes, i have own fork with that modification so i will make PR when i will been ready.

wstrange commented 9 months ago

I have an experiment in branch https://github.com/wstrange/asn1lib/tree/longtags_65 that parses the sample cert you provided as an ASN1Object with an extendedTag property . You need to interpret the object.valueBytes() as the library doesn't know what to do with this object type.

Does this help at all?

See https://github.com/wstrange/asn1lib/blob/longtags_65/test/issue_65_long_tags_test.dart

(I'm not sure this is correct yet.. so needs validation)

wstrange commented 9 months ago

I pushed 1.5.2, which provides some basic support for parsing extended tags. If this is not sufficient, feel free to re-open this, or create another issue

wstrange / asn1lib