search

P1sec / pycrate

A Python library to ease the development of encoders and decoders for various protocols and file formats; contains ASN.1 and CSN.1 compilers.
GNU Lesser General Public License v2.1
381 stars 132 forks source link

get_at is not working correctly for SEQ OF as it always returns information for last instance. #203

Closed dbressan2 closed 2 years ago

dbressan2 commented 2 years ago

Hi

get_at always return the last instance of a SEQ OF, even though the instance is explicitly mentioned in the path and does not correspond to last instance.

As an example, we try to access second item here starting with E120 ..

dcch.get_at(['message', 'c1', 'ueCapabilityInformation', 'criticalExtensions', 'c1', 'ueCapabilityInformation-r8', 'ue-CapabilityRAT-ContainerList', 1, 'ueCapabilityRAT-Container']).get_val().hex()

But last octet string is returned: Out[83]: 'a107c04a160401c28000000440040140280b003a030a742d8e7c3100008000802020080280'

We would expect E12010 ... Note: get_val_at is working properly but does not allow to access the ASN.1 object, just its value.

{ message c1 : ueCapabilityInformation : { rrc-TransactionIdentifier 3, criticalExtensions c1 : ueCapabilityInformation-r8 : { ue-CapabilityRAT-ContainerList { -- Item 0 -- { rat-Type eutra, ueCapabilityRAT-Container 'DDB80'H }, -- Item 1 -- { rat-Type nr, ueCapabilityRAT-Container 'E1201000 ... 'H }, -- Item 2 -- { rat-Type eutra-nr, ueCapabilityRAT-Container 'A107C04A160401C28000000440040140280B003A030A742D8E7C3100008000802020080280'H } } } } }

Many thanks in advance

Full code: message0 = """ 3e 03 08 19 2d db 80 50 30 9f bf 7e 0c 1f 83 f0 7e 0f df 83 76 26 80 08 0d cf 64 00 00 00 01 78 68 0c 0a 0c 77 c0 00 00 00 b2 f0 03 00 24 10 86 00 60 84 30 03 00 30 40 18 20 0c 20 84 30 42 18 21 0c 10 86 08 43 00 30 80 10 c2 00 43 04 01 82 00 c0 0c 00 c0 09 68 3e aa 30 e0 b8 18 40 00 82 07 f8 00 20 80 fe 10 00 ff 00 04 00 70 80 01 05 08 20 81 84 00 09 20 ff 80 02 0b f8 41 03 fc 00 12 40 ff 00 04 50 7f c0 01 01 fc 20 82 fe 10 3f c0 01 04 2f f0 00 41 7f 08 2f e1 04 1f f0 00 49 7f 08 20 bf 84 9f f0 00 41 07 fc 00 11 5f c2 08 2f e1 17 fc 00 10 40 ff 00 04 d0 bf c0 01 01 fc 20 bf 84 10 9f c2 07 f8 00 20 bf 84 10 7f c0 01 01 fc 24 82 fe 10 3f c0 01 24 0c 20 10 49 03 fc 04 12 40 ff 00 84 52 e0 fc 1f 83 f0 7e 0f c1 f8 3f 07 e0 fc 1f 83 f0 7e 0f c1 f8 3f 07 e0 fc 1f 83 f0 7e 0f c1 f8 3f 07 ed 9d 00 00 11 97 08 b8 00 00 e5 c0 00 40 40 40 80 41 81 00 00 50 14 04 04 04 04 04 14 19 01 05 06 40 42 40 50 14 04 14 05 01 05 01 40 41 40 50 10 50 14 04 04 04 24 19 01 40 42 41 90 14 04 14 19 01 05 06 40 40 00 40 40 40 66 80 03 e6 57 9a 04 01 48 7f f9 a5 2e f1 c0 3a 00 16 08 00 64 92 00 00 40 00 a2 09 73 40 00 a1 e1 00 00 00 21 00 00 01 02 08 20 80 00 20 82 18 20 82 08 20 80 18 00 00 00 51 90 00 00 00 40 02 20 10 50 42 05 81 0c e1 20 10 00 07 4f 5a 13 02 00 02 01 40 48 24 c5 80 02 60 d4 0c 13 60 18 73 e6 18 f2 85 81 c1 30 08 0c 41 00 3a 0d 85 18 1a c1 18 43 41 02 0f 94 02 2c b6 4c 55 0d 0e 7c 47 1e 81 9f 60 4f 8c 73 e6 18 f2 85 81 c0 e1 e0 b8 00 27 80 0e 00 09 e0 00 38 01 c0 00 20 0a 01 47 27 80 22 03 01 f0 28 01 00 00 00 00 80 00 01 00 40 00 01 00 20 00 00 c0 10 00 00 60 08 00 00 40 04 00 00 28 02 00 00 18 01 00 00 04 00 80 00 07 00 40 00 04 01 0c a9 bc a9 ac 0c ac 0d ac 99 ac a5 ac 8d ac 04 ac 08 a0 48 20 00 01 2c a4 a0 80 00 0c b2 92 82 00 00 52 ca 48 08 00 01 cb 29 20 20 00 09 2c a4 80 80 00 2c b2 92 02 00 00 d2 ca 48 08 00 03 cb 29 20 20 00 11 2c a4 80 80 00 4c b2 92 01 2a 55 57 4a aa a8 1c 55 2a 8b a5 51 54 9a 2a 94 c5 52 38 aa 03 15 40 a2 80 87 40 2a 81 40 e0 70 28 14 0a 05 02 81 40 a0 08 50 18 12 0a 12 32 00 62 5a 10 7c 04 a1 60 40 1c 28 00 00 00 44 00 40 14 02 80 b0 03 a0 30 a7 42 d8 e7 c3 10 00 08 00 08 02 02 00 80 28 00 """.replace(" ", "").replace("\n", "") dcch = lte_g80.EUTRA_RRC_Definitions.UL_DCCH_Message dcch.from_uper(bytes.fromhex(message0))

print (dcch.get_at(['message', 'c1', 'ueCapabilityInformation', 'criticalExtensions', 'c1', 'ueCapabilityInformation-r8', 'ue-CapabilityRAT-ContainerList', 1, 'ueCapabilityRAT-Container']).get_val().hex())

p1-bmu commented 2 years ago

I am sorry, I don't really understand what is the issue. ASN.1 objects and values must be handled separately, they are different things : the item within a SEQUENCE OF (or SET OF) is a single object, and it is used to encode / decode several different values. When you say you want to access the 2nd item of a SEQUENCE OF, this is related to the value, not the object.

If you want to get the 2nd of 3rd item value associated to its original object, you need to select both independently, and reassign the value to the object, e.g.:

obj = dcch.get_at(['message', 'c1', 'ueCapabilityInformation', 'criticalExtensions', 'c1', 'ueCapabilityInformation-r8', 'ue-CapabilityRAT-ContainerList', None, 'ueCapabilityRAT-Container'])
val = dcch.get_at(['message', 'c1', 'ueCapabilityInformation', 'criticalExtensions', 'c1', 'ueCapabilityInformation-r8', 'ue-CapabilityRAT-ContainerList', 1, 'ueCapabilityRAT-Container'])

obj.set_val(val)

Moreover, it would be greatly appreciated if you can do some text formatting effort when filling an issue. Thanks.

dbressan2 commented 2 years ago

Hi

The problem is that a ue-CapabilityRAT-ContainerList contains several instances of rat-Type and ueCapabilityRAT-Container, and I want to decode each of them according to the correct grammar depending on RAT type. Namely:

I know how to do this with asn1c and with Wireshark, but I cannot find a solution with Pycrate.

The following code still fails unfortunately (full code attached):

dcch = lte_g80.EUTRA_RRC_Definitions.UL_DCCH_Message dcch.from_uper(bytes.fromhex(message0))

container_list = ['message', 'c1', 'ueCapabilityInformation', 'criticalExtensions', 'c1', 'ueCapabilityInformation-r8', 'ue-CapabilityRAT-ContainerList']

print (dcch.to_asn1())

num_tech = 3 extensions = [] for tech in range(num_tech): container_obj = dcch.get_at(container_list + [None, 'ueCapabilityRAT-Container']) container_val = dcch.get_val_at(container_list + [tech, 'ueCapabilityRAT-Container']) container_obj.set_val(container_val)

tech_rat_container = dcch.get_val_at(container_list + [tech])
rat_info = tech_rat_container['rat-Type']
container = tech_rat_container['ueCapabilityRAT-Container']

if rat_info == "eutra":

    _cont_UE_EUTRA_Capability = SEQ(name='UE-EUTRA-Capability', mode=MODE_TYPE,
                                  typeref=ASN1RefType(('EUTRA-RRC-Definitions', 'UE-EUTRA-Capability')))

    container_obj._const_cont =  _cont_UE_EUTRA_Capability
    container_obj._const_cont_enc = None
    extensions = [_cont_UE_EUTRA_Capability]

    lte_g80.EUTRA_RRC_Definitions._all_.extend(extensions)
    init_modules(lte_g80.EUTRA_RRC_Definitions)
    dcch.from_uper(bytes.fromhex(message0))

Thank you very much for your help

Best regards

dB

From: Benoit Michau @.**@.> Sent: Monday, June 27, 2022 12:02 PM To: P1sec/pycrate @.**@.> Cc: Dominique Bressanelli @.**@.>; Author @.**@.> Subject: Re: [P1sec/pycrate] get_at is not working correctly for SEQ OF as it always returns information for last instance. (Issue #203)

WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

I am sorry, I don't really understand what is the issue. ASN.1 objects and values must be handled separately, they are different things : the item within a SEQUENCE OF (or SET OF) is a single object, and it is used to encode / decode several different values. When you say you want to access the 2nd item of a SEQUENCE OF, this is related to the value, not the object.

If you want to get the 2nd of 3rd item value associated to its original object, you need to select both independently, and reassign the value to the object, e.g.:

obj = dcch.get_at(['message', 'c1', 'ueCapabilityInformation', 'criticalExtensions', 'c1', 'ueCapabilityInformation-r8', 'ue-CapabilityRAT-ContainerList', None, 'ueCapabilityRAT-Container'])

val = dcch.get_at(['message', 'c1', 'ueCapabilityInformation', 'criticalExtensions', 'c1', 'ueCapabilityInformation-r8', 'ue-CapabilityRAT-ContainerList', 1, 'ueCapabilityRAT-Container'])

obj.set_val(val)

Moreover, it would be greatly appreciated if you can do some text formatting effort when filling an issue. Thanks.

— Reply to this email directly, view it on GitHubhttps://github.com/P1sec/pycrate/issues/203#issuecomment-1167149072, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHUUMPLXR2GPMPFHKCU5XK3VRF32PANCNFSM5ZPKCLHA. You are receiving this because you authored the thread.Message ID: @.**@.>>

p1-bmu commented 2 years ago

In what you describe, the ueCapabilityRAT-Container is a different object depending of rat-Type. I would personally not try to mess with the ASN.1 schema the way you do, as supporting properly those distinct objects conditionally in ASN.1 requires generally to define table constraints in UE-CapabilityRAT-Container. Something in this fashion could do the job, I suppose:

RRCLTE-RAT-CONTAINER ::= CLASS {
    &rat-Type RAT-Type UNIQUE,
    &Container
}
WITH SYNTAX {
    TYPE &rat-Type
    CONTAINER &Container
}

RRCLTE-RAT-Containers RRCLTE-RAT-CONTAINER ::= {
    {TYPE "eutra" CONTAINER UE-EUTRA-Capability}|
    -- {define here other types - container relationships}| --
    ...
}

UE-CapabilityRAT-Container ::= SEQUENCE {
    rat-Type RRCLTE-RAT-CONTAINER.&rat-Type ({RRCLTE-RAT-Containers}),
    ueCapabilityRAT-Container RRCLTE-RAT-CONTAINER.&Container ({RRCLTE-RAT-Containers}{@rat-Type})
}

On my side, I would rather go for programmatically decode those distinct containers with a custom function, after the parent RRC message is decoded, and not try to build an over-engineered ASN.1 schema.

dbressan2 commented 2 years ago

Thank you

I fully understand your explanations. Unfortunately, 3GPP often uses the case where OCTET STRING decoding depends on Rat-Type. I settled for a solution with multiple ASN.1 decoding for containers after decoding parent object, as you suggested.

We can close this thread now. I would still think it may be handy to get a way to iterate through a SEQ OF and be able to use get_at for each item of the SEQ, but again I understood why it would not work currently.

Thank you very much for your help

Best regards

Dominique

From: Benoit Michau @.> Sent: Tuesday, June 28, 2022 12:33 PM To: P1sec/pycrate @.> Cc: Dominique Bressanelli @.>; Author @.> Subject: Re: [P1sec/pycrate] get_at is not working correctly for SEQ OF as it always returns information for last instance. (Issue #203)

WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

In what you describe, the ueCapabilityRAT-Container is a different object depending of rat-Type. I would personally not try to mess with the ASN.1 schema the way you do, as supporting properly those distinct objects conditionally in ASN.1 requires generally to define table constraints in UE-CapabilityRAT-Container. Something in this fashion could do the job, I suppose:

RRCLTE-RAT-CONTAINER ::= CLASS {

    &rat-Type RAT-Type UNIQUE,

    &Container

}

WITH SYNTAX {

    TYPE &rat-Type

    CONTAINER &Container

}

RRCLTE-RAT-Containers RRCLTE-RAT-CONTAINER ::= {

    {TYPE "eutra" CONTAINER UE-EUTRA-Capability}|

    -- {define here other types - container relationships}| --

    ...

}

UE-CapabilityRAT-Container ::= SEQUENCE {

    rat-Type RRCLTE-RAT-CONTAINER.&rat-Type ({RRCLTE-RAT-Containers}),

    ueCapabilityRAT-Container RRCLTE-RAT-CONTAINER.&Container ***@***.***})

}

On my side, I would rather go for programmatically decode those distinct containers with a custom function, after the parent RRC message is decoded, and not try to build an over-engineered ASN.1 schema.

— Reply to this email directly, view it on GitHubhttps://github.com/P1sec/pycrate/issues/203#issuecomment-1168545914, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHUUMPLCVT3T5SOF2ZUF3PTVRLIG3ANCNFSM5ZPKCLHA. You are receiving this because you authored the thread.Message ID: @.**@.>>

p1-bmu commented 2 years ago

Here is an alternative way to process sub-objects, by working with the JSON ASN.1 encoding and the jsonpath_ng library: https://github.com/P1sec/pycrate/issues/206#issuecomment-1189199684