Long-From of `Identifier octets`

bobaxix commented 10 months ago

Description

BER encoding defines, that ASN1 tags can be encoded as 1 or more bytes (according to Tag Type value)

https://en.wikipedia.org/wiki/X.690#BER_encoding

It was intentional to use as tag always first byte of encodedBytes?

https://github.com/wstrange/asn1lib/blob/master/lib/src/asn1parser.dart#L29C62 https://github.com/wstrange/asn1lib/blob/master/lib/src/asn1object.dart#L71

To reproduce

import 'dart:typed_data';
import 'package:asn1lib/asn1lib.dart';

// Tag is 0x5F21, what means:
//
// - Tag class: 0x01 (Application)
// - P/C: 0 (Primitive)
// - Tag type: 0x21
final values= Uint8List.fromList(
  [
    0x5F,
    0x21,
    0x81,
    0x01,
    0x7F,
  ],
);

void main() {
  var asn1Parser = ASN1Parser(
    values,
  );

  //Throws:
  //
  //Unhandled exception:
  //Instance of 'ASN1Exception'
  //#0      ASN1Parser._doPrimitive (package:asn1lib/src/asn1parser.dart:136:9)
  //#1      ASN1Parser.nextObject (package:asn1lib/src/asn1parser.dart:56:15)
  //#2      main (file:///C:/Users/marcinb/Desktop/testx/bin/testx.dart:19:20)
  //#3      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:296:19)
  //#4      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:189:12)
  print(asn1Parser.nextObject());
}

wstrange commented 10 months ago

Not 100% sure here. Shot in the dark..

Where are those bytes coming from? It kinda look like the first byte (0x5F) is not part of the asn1 stream?

If I comment out that first byte, and enable relaxed parsing (i.e. tells the parser to parse unknown application types as a plain old ASN1Object). It kinda works:

import 'dart:typed_data';
import 'package:asn1lib/asn1lib.dart';

// Tag is 0x5F21, what means:
//
// - Tag class: 0x01 (Application)
// - P/C: 0 (Primitive)
// - Tag type: 0x21
final values= Uint8List.fromList(
  [
    // 0x5F,
    0x21,
    0x81,
    0x01,
    0x7F,
  ],
);

void main() {
  var asn1Parser = ASN1Parser(
    values,
    relaxedParsing: true,
  );

  //Throws:
  //
  //Unhandled exception:
  //Instance of 'ASN1Exception'
  //#0      ASN1Parser._doPrimitive (package:asn1lib/src/asn1parser.dart:136:9)
  //#1      ASN1Parser.nextObject (package:asn1lib/src/asn1parser.dart:56:15)
  //#2      main (file:///C:/Users/marcinb/Desktop/testx/bin/testx.dart:19:20)
  //#3      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:296:19)
  //#4      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:189:12)
  print(asn1Parser.nextObject());
}

Gives me the output:

ASN1Object(tag=21 valueByteLength=1) startpos=3 bytes=[0x21, 0x81, 0x1, 0x7f]

wstrange commented 10 months ago

Also, are there possibly extra bytes needed in that stream? If I run the debugger on your original bytes, the length is calculated as 35 bytes. That is past the end of that stream.

bobaxix commented 10 months ago

@wstrange

Comments

Sequence is dummy - dont be particular about it.
Look there https://luca.ntop.org/Teaching/Appunti/asn1.html (document from README.md)

Short encoding:

Low-tag-number form. One octet. Bits 8 and 7 specify the class (see Table 2), bit 6 has value "0," indicating that the encoding is primitive, and bits 5-1 give the tag number.

Long encoding:

High-tag-number form. Two or more octets. First octet is as in low-tag-number form, except that bits 5-1 all have value "1." Second and following octets give the tag number, base 128, most significant digit first, with as few digits as possible, and with the bit 8 of each octet except the last set to "1."

I know that "simple types" of ASN.1 contains only this values. In one byte tag we have 5 bits for tag number, so we can encode values from 0 to 31

But from this ASN1 contains also values above this range (example: DURATION (34₁₀, 22₁₆). How you encode it on five bits?

Real life example

I working with tachographs, and there are special cards to store information about company/driver etc. Cards are protected via certification system which use certificate chains. Certificate always starts with application specific sequence tag 7F 21 [len].

In your lib I cannot read sequence because 21 is treat as length, not [len] field.

Post-scriptum

I'm not expert in ASN1 and BER so maybe something is missing in my mind. I also check another lib to ASN1 encoding (https://pub.dev/packages/pointycastle) and there also only one byte is used as tag, so it suggest my error.

wstrange commented 10 months ago

I think it would help to provide a proper ASN1 sequence (not dummy bytes). We can further diagnose the problem.

bobaxix commented 10 months ago

@wstrange especially for you:

final certificate = Uint8List.fromList(
[
    0x7F, 0x21, 0x81, 0xC9, 0x7F, 0x4E, 0x81, 0x82, 0x5F, 0x29, 0x01, 0x00, 0x42, 0x08, 0xFD, 0x45,
    0x43, 0x20, 0x01, 0xFF, 0xFF, 0x01, 0x5F, 0x4C, 0x07, 0xFF, 0x53, 0x4D, 0x52, 0x44, 0x54, 0x0D,
    0x7F, 0x49, 0x4E, 0x06, 0x09, 0x2B, 0x24, 0x03, 0x03, 0x02, 0x08, 0x01, 0x01, 0x07, 0x86, 0x41,
    0x04, 0x08, 0xC0, 0x4E, 0x39, 0x26, 0xC8, 0xDE, 0x85, 0x54, 0x42, 0x40, 0xCD, 0xE4, 0x0D, 0xAB,
    0x70, 0xD2, 0xB4, 0x7E, 0x0F, 0x83, 0x76, 0x25, 0x22, 0xD7, 0xB0, 0xB8, 0x54, 0x3B, 0x9B, 0x29,
    0xDC, 0x80, 0xE5, 0xC6, 0x7B, 0x82, 0xA6, 0x2D, 0x55, 0xE3, 0x48, 0x3A, 0xB4, 0xB0, 0x0A, 0x24,
    0xC2, 0xA2, 0x56, 0x6C, 0x37, 0x86, 0x79, 0x7A, 0x1A, 0x05, 0x28, 0x22, 0xAB, 0x4B, 0xF1, 0xF2,
    0x92, 0x5F, 0x20, 0x08, 0xFD, 0x45, 0x43, 0x20, 0x01, 0xFF, 0xFF, 0x01, 0x5F, 0x25, 0x04, 0x5B,
    0x21, 0xB0, 0x00, 0x5F, 0x24, 0x04, 0x9B, 0x8F, 0xAE, 0x80, 0x5F, 0x37, 0x40, 0x65, 0xC6, 0x2A,
    0xC1, 0x3D, 0xED, 0x14, 0x7F, 0xA8, 0xD1, 0xD1, 0x1A, 0x8F, 0x5B, 0xF2, 0xCF, 0x9E, 0x95, 0xDB,
    0x1B, 0x43, 0xD2, 0x53, 0xB4, 0x8B, 0x61, 0x5B, 0x2F, 0xE7, 0x0B, 0x3F, 0xD8, 0x2A, 0xA8, 0xD3,
    0x3D, 0x27, 0xF0, 0xF4, 0xD7, 0x36, 0x7C, 0x04, 0x90, 0x3B, 0xBB, 0xE6, 0x37, 0x5B, 0x64, 0x3A,
    0x19, 0xC5, 0xB8, 0x3D, 0x19, 0xFC, 0x74, 0x85, 0xDB, 0x47, 0x6C, 0x70, 0x67,
  ],
);

You can download it from here ERCA Gen2 (1) Root Certificate.bin

wstrange commented 10 months ago

Is that an x509 certificate? Have you looked at https://pub.dev/packages/x509 for parsing it?

How sure are you that all of those bytes are ASN1 encoded ?

wstrange commented 10 months ago

FWIW, openssl does not think that PEM file is a certificate. I'm really lost on what format that root cert is supposed to be.

➜ cert openssl x509 -in t.pem -text Could not find certificate from t.pem

Is there another utility or language library that correctly parses that PEM file?

bobaxix commented 10 months ago

Why do you assume that is X509 certificate ?

Please dont focus on data type, using ASN1 you can encode any data. One and only question is that tag in TLV can be more that 1 byte.

Here, look for CSM_134 „this” certificate content is described.

wstrange commented 10 months ago

Do you have an example of another library or CLI utility that can parse that data? i.e. so we can compare

wstrange commented 10 months ago

None of the previous applications of this library need more than one byte for the tag. So if this format needs > 1 it is going to be a fairly large change.

I'm not sure when I'd get to this. If you'd consider a PR I would take a look

bobaxix commented 10 months ago

Yes, i have own fork with that modification so i will make PR when i will been ready.

wstrange commented 10 months ago

I have an experiment in branch https://github.com/wstrange/asn1lib/tree/longtags_65 that parses the sample cert you provided as an ASN1Object with an extendedTag property . You need to interpret the object.valueBytes() as the library doesn't know what to do with this object type.

Does this help at all?

See https://github.com/wstrange/asn1lib/blob/longtags_65/test/issue_65_long_tags_test.dart

(I'm not sure this is correct yet.. so needs validation)

wstrange commented 10 months ago

I pushed 1.5.2, which provides some basic support for parsing extended tags. If this is not sufficient, feel free to re-open this, or create another issue

wstrange / asn1lib