search

a-sit-plus / signum

Kotlin Multiplatform Crypto/PKI Library and ASN1 Parser + Encoder
https://a-sit-plus.github.io/signum/
Apache License 2.0
76 stars 6 forks source link

Add Support for CONTEXT-SPECIFIC, APPLICATION, and PRIVATE Tags #116

Closed ShiinaSekiu closed 2 months ago

ShiinaSekiu commented 2 months ago

I am trying to use signum to parse ASN.1 data from SGP.22 However, it seems that SGP.22 uses a lot of CONTEXT-SPECIFIC and APPLICATION tags. It appears that signum is not compatible with these. Will there be consideration for supporting these tags in the future? Thank you. Sample data:

BF 3E 12 5A 10 89 08 60 30 20 22 00 00 00 24 00 00 30 97 96 56

Format:

GetEuiccDataResponse ::= [62] SEQUENCE { -- Tag 'BF3E'
    eidValue [APPLICATION 26] Octet16  -- tag '5A'
}

Octet16 ::= OCTET STRING (SIZE(16))

Decode online

JesusMcCloud commented 2 months ago

Thank you for reporting this issue! It is indeed a bug in the parser and we'll fix it.

ShiinaSekiu commented 2 months ago

By the way, will signum in the future have a tool similar to ASN1bean to generate model classes from ASN format files?

JesusMcCloud commented 2 months ago

We currently have no such plans

ShiinaSekiu commented 2 months ago

Thanks for the reply, looking forward to the fix.

JesusMcCloud commented 2 months ago

initial work started here: https://github.com/a-sit-plus/signum/tree/feature/asn1ProperTagging to get multi-byte tags going.

JesusMcCloud commented 2 months ago

@ShiinaSekiu the new WIP parser will give you the following for your example:

 Tagged(tag=CONTEXT_SPECIFIC 0x3E CONSTRUCTED, length=18, overallLength=21)
   {
     Primitive(tag=APPLICATION 0x1A, length=16, overallLength=18, content=89086030202200000024000030979656)
   }
ShiinaSekiu commented 2 months ago

Wow this looks great, thanks for the work. Please give me a code sample when you finish, thx.

JesusMcCloud commented 2 months ago

@ShiinaSekiu see #117. We'd very much appreciate test data (DER-encoded byte strings are sufficient, we can test compliance against Bouncy Castle's ASN.1 parser/encoder)

JesusMcCloud commented 2 months ago

@ShiinaSekiu we will merge #117 next week and tackle #13 before the next release too, but it should only be a matter of days

ShiinaSekiu commented 2 months ago

Something seems to have gone wrong. It doesn't parse correctly. Code Sample:

    val element = Asn1.Tagged(62UL) {
        +Asn1Primitive(Asn1Element.Tag(26UL, false, TagClass.APPLICATION), "89086030202200000024000030979656".hexToByteArray())
    }
    val derHexString = element.toDerHexString()
    println(derHexString)
    println(derHexString == "BF3E125A1089086030202200000024000030979656")
    println(Asn1Element.decodeFromDerHexString(derHexString))

Output:

BF3E125A1089086030202200000024000030979656
true
Exception in thread "main" at.asitplus.signum.indispensable.asn1.Asn1Exception: Out of bytes to decode
    at at.asitplus.signum.indispensable.asn1.Asn1DecodingKt.readTlv(Asn1Decoding.kt:710)
    at at.asitplus.signum.indispensable.asn1.Asn1DecodingKt.access$readTlv(Asn1Decoding.kt:1)
    at at.asitplus.signum.indispensable.asn1.Asn1Reader.read(Asn1Decoding.kt:68)
    at at.asitplus.signum.indispensable.asn1.Asn1Reader.doParse(Asn1Decoding.kt:39)
    at at.asitplus.signum.indispensable.asn1.Asn1DecodingKt.parse(Asn1Decoding.kt:26)
    at at.asitplus.signum.indispensable.asn1.Asn1Element$Companion.decodeFromDerHexString(Asn1Elements.kt:48)
    at SGPTestKt.main(SGPTest.kt:15)
    at SGPTestKt.main(SGPTest.kt)
Caused by: java.lang.IllegalArgumentException: Out of bytes to decode
    at at.asitplus.signum.indispensable.asn1.Asn1DecodingKt.readTlv(Asn1Decoding.kt:324)
    ... 7 more
ShiinaSekiu commented 2 months ago

Additionally, I propose we introduce an API akin to JsonElement from kotlinx.serialization, enabling swift and direct casting to subtype elements. It should look like: Asn1Element.structure, Asn1Element.primitive etc. References: Element to Object Conversion Element to Array Conversion

ShiinaSekiu commented 2 months ago

Here is a larger ASN1 data for testing

JesusMcCloud commented 2 months ago

Something seems to have gone wrong. It doesn't parse correctly. Code Sample:

    val element = Asn1.Tagged(62UL) {
        +Asn1Primitive(Asn1Element.Tag(26UL, false, TagClass.APPLICATION), "89086030202200000024000030979656".hexToByteArray())
    }
    val derHexString = element.toDerHexString()
    println(derHexString)
    println(derHexString == "BF3E125A1089086030202200000024000030979656")
    println(Asn1Element.decodeFromDerHexString(derHexString))

Output:

BF3E125A1089086030202200000024000030979656
true
Exception in thread "main" at.asitplus.signum.indispensable.asn1.Asn1Exception: Out of bytes to decode
  at at.asitplus.signum.indispensable.asn1.Asn1DecodingKt.readTlv(Asn1Decoding.kt:710)
  at at.asitplus.signum.indispensable.asn1.Asn1DecodingKt.access$readTlv(Asn1Decoding.kt:1)
  at at.asitplus.signum.indispensable.asn1.Asn1Reader.read(Asn1Decoding.kt:68)
  at at.asitplus.signum.indispensable.asn1.Asn1Reader.doParse(Asn1Decoding.kt:39)
  at at.asitplus.signum.indispensable.asn1.Asn1DecodingKt.parse(Asn1Decoding.kt:26)
  at at.asitplus.signum.indispensable.asn1.Asn1Element$Companion.decodeFromDerHexString(Asn1Elements.kt:48)
  at SGPTestKt.main(SGPTest.kt:15)
  at SGPTestKt.main(SGPTest.kt)
Caused by: java.lang.IllegalArgumentException: Out of bytes to decode
  at at.asitplus.signum.indispensable.asn1.Asn1DecodingKt.readTlv(Asn1Decoding.kt:324)
  ... 7 more

What happens here is: The input contains trailing bytes. The first element is parsed correctly, then the junk bytes are being parsed and this fails. I've created a separate issue #121, as it is unrelated to tagging.

EDIT: My bad! copied the wrong hex string!

JesusMcCloud commented 2 months ago

Additionally, I propose we introduce an API akin to JsonElement from kotlinx.serialization, enabling swift and direct casting to subtype elements. It should look like: Asn1Element.structure, Asn1Element.primitive etc. References: Element to Object Conversion Element to Array Conversion

Created #122 for this matter

JesusMcCloud commented 2 months ago

@ShiinaSekiu the lastest commit in feature/asn1ProperTagging fixed the parsing issues

ShiinaSekiu commented 2 months ago

@ShiinaSekiu the lastest commit in feature/asn1ProperTagging fixed the parsing issues

Thanks, I received the update and after many tests it doesn't seem to have any problems, I will test more!

ShiinaSekiu commented 2 months ago

Data Sample Format:

-- Definition of ProfileInfoListResponse
ProfileInfoListResponse ::= [45] CHOICE { -- Tag 'BF2D'
profileInfoListOk SEQUENCE OF ProfileInfo,
profileInfoListError ProfileInfoListError
}
ProfileInfo ::= [PRIVATE 3] SEQUENCE { -- Tag 'E3'
iccid Iccid OPTIONAL,
isdpAid [APPLICATION 15] OctetTo16 OPTIONAL, -- AID of the ISD-P containing the 
Profile, tag '4F'profileState [112] ProfileState OPTIONAL, -- Tag '9F70'
profileNickname [16] UTF8String (SIZE(0..64)) OPTIONAL, -- Tag '90'
serviceProviderName [17] UTF8String (SIZE(0..32)) OPTIONAL, -- Tag '91'
profileName [18] UTF8String (SIZE(0..64)) OPTIONAL, -- Tag '92'
iconType [19] IconType OPTIONAL, -- Tag '93'
icon [20] OCTET STRING (SIZE(0..1024)) OPTIONAL, -- Tag '94', see condition in 
ES10c:GetProfilesInfo
profileClass [21] ProfileClass OPTIONAL, –- Tag '95'
notificationConfigurationInfo [22] SEQUENCE OF 
NotificationConfigurationInformation OPTIONAL, -- Tag 'B6'
profileOwner [23] OperatorId OPTIONAL, -- Tag 'B7'
dpProprietaryData [24] DpProprietaryData OPTIONAL, -- Tag 'B8'
profilePolicyRules [25] PprIds OPTIONAL –- Tag '99'}
IconType ::= INTEGER {jpg(0), png(1)}
ProfileState ::= INTEGER {disabled(0), enabled(1)}
ProfileClass ::= INTEGER {test(0), provisioning(1), operational(2)}
ProfileInfoListError ::= INTEGER {incorrectInputValues(1), undefinedError(127)}

Parse Code:

    val asn1 = Asn1Element.decodeFromDerHexString("BF 2D 82 01 0C A0 82 01 08 E3 82 01 04 5A 0A 98 05 03 03 03 19 09 08 36 F9 4F 10 A0 00 00 05 59 10 10 FF FF FF FF 89 00 00 12 00 9F 70 01 00 91 04 54 69 67 6F 92 04 54 69 67 6F 93 01 00 94 81 CD 89 50 4E 47 0D 0A 1A 0A 00 00 00 0D 49 48 44 52 00 00 00 40 00 00 00 40 08 02 00 00 00 25 0B E6 89 00 00 00 01 73 52 47 42 00 AE CE 1C E9 00 00 00 04 67 41 4D 41 00 00 B1 8F 0B FC 61 05 00 00 00 09 70 48 59 73 00 00 0E C3 00 00 0E C3 01 C7 6F A8 64 00 00 00 62 49 44 41 54 68 43 ED CF B1 0D C0 30 0C C0 30 37 FF FF EC 2E F9 81 08 20 2E 9A F5 ED EE BC EC DC 3E AB 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 6B E6 07 72 AA 03 7D DD F3 53 ED 00 00 00 00 49 45 4E 44 AE 42 60 82 95 01 02")
    val profile = ((asn1 as Asn1Structure).children[0] as Asn1Structure).children[0] as Asn1Structure
    val profileName = (profile.children.first { it.tag.tagValue == 18UL } as Asn1Primitive).readString()
    println(profileName)

Error Info:

Exception in thread "main" at.asitplus.signum.indispensable.asn1.Asn1Exception: Input contains invalid chars: 'Tigo'
    at at.asitplus.signum.indispensable.asn1.Asn1String$Numeric.<init>(Asn1String.kt:94)
    at at.asitplus.signum.indispensable.asn1.Asn1DecodingKt.readString(Asn1Decoding.kt:154)
    at SGPTestKt.main(SGPTest.kt:211)
    at SGPTestKt.main(SGPTest.kt)

In this case, I'm trying to resolve the profileName value of the ProfileInfo object. But since this is a CONTEXT_SPECIFIC tag, it is not declared as a String tag and therefore cannot be called readString(), do you have any idea or should we exempt type checking when reading these tags? Similar problems exist with other types, such as readInt().

JesusMcCloud commented 2 months ago

you can always fetch the content's of the primitive by calling .content and then manually call the decoding functions that operate on raw bytes:

val asn1 = Asn1Element.decodeFromDerHexString("BF 2D 82 01 0C A0 82 01 08 E3 82 01 04 5A 0A 98 05 03 03 03 19 09 08 36 F9 4F 10 A0 00 00 05 59 10 10 FF FF FF FF 89 00 00 12 00 9F 70 01 00 91 04 54 69 67 6F 92 04 54 69 67 6F 93 01 00 94 81 CD 89 50 4E 47 0D 0A 1A 0A 00 00 00 0D 49 48 44 52 00 00 00 40 00 00 00 40 08 02 00 00 00 25 0B E6 89 00 00 00 01 73 52 47 42 00 AE CE 1C E9 00 00 00 04 67 41 4D 41 00 00 B1 8F 0B FC 61 05 00 00 00 09 70 48 59 73 00 00 0E C3 00 00 0E C3 01 C7 6F A8 64 00 00 00 62 49 44 41 54 68 43 ED CF B1 0D C0 30 0C C0 30 37 FF FF EC 2E F9 81 08 20 2E 9A F5 ED EE BC EC DC 3E AB 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 AD 01 6B E6 07 72 AA 03 7D DD F3 53 ED 00 00 00 00 49 45 4E 44 AE 42 60 82 95 01 02")
val profile = ((asn1 as Asn1Structure).children[0] as Asn1Structure).children[0] as Asn1Structure
val profileName = (profile.children.first { it.tag.tagValue == 18UL } as Asn1Primitive).content.decodeToString()

in cases where the default tags deviate, you can always provider your own decoding function (see https://a-sit-plus.github.io/signum/indispensable/at.asitplus.signum.indispensable.asn1/decode.html). Of course, the current development branch now supports passing Tag objects instead of numbers.

All low-level decoding functions are in Asn1Decoding.kt. For numbers you can use decodeFromDerValue (see https://github.com/a-sit-plus/signum/blob/development/indispensable/src/commonMain/kotlin/at/asitplus/signum/indispensable/asn1/Asn1Decoding.kt#L295).

A more coherent naming of decoding functions would make such tasks much easier. I opened issue #125 for this

ShiinaSekiu commented 2 months ago

Yes, I've found the low-level decoding functions. And I'm looking forward to your optimization of the API structure.

JesusMcCloud commented 2 months ago

@ShiinaSekiu close this at your discretion

JesusMcCloud commented 2 months ago

@ShiinaSekiu 4.0.0-SNAPSHOT of branch feature/consistentAPI just got published (see README/CHANGELOG) regarding encoding/decoding API revamp)

ShiinaSekiu commented 2 months ago

@ShiinaSekiu close this at your discretion

I'm sorry I'm late. I've been busy with other things lately, and I've been using a lot of data to test the parser and it hasn't had any problems. If I find a new one I'll open a new issue. Thanks for your work!