Extending Information Object Sets at runtime

[benmaddison]

I am trying to decode DER objects that have open-type components, and where the constraining information content set is extended by a separate ASN.1 module.

I can't seem to find a way to get the ASN.1 runtime to recognise the additional information object instance as a member of the information object set.

For example, an ASN.1 module defines and uses an extensible object information class and set like so:

FooModule

DEFINITIONS IMPLICIT TAGS ::=
BEGIN

FOO-TYPE ::= CLASS {
    &id INTEGER UNIQUE,
    &Foo OPTIONAL
} WITH SYNTAX {
    [FOO &Foo] IDENTIFIED BY &id
}

Bar ::= OCTET STRING

foo-Bar FOO-TYPE ::= {
    FOO Bar IDENTIFIED BY 1
}

FooSet FOO-TYPE ::= {
    foo-Bar,
    ...
}

FooThing ::= SEQUENCE {
    id FOO-TYPE.&id({FooSet})
    fooContent [0] EXPLICIT FOO-TYPE.&Foo({FooSet}{@id})
}

And then in a different module, another FOO-TYPE is defined:

BazModule

DEFINITIONS IMPLICIT TAGS ::=
BEGIN

IMPORTS
    FOO-TYPE FROM FooModule

Baz ::= INTEGER

foo-Baz FOO-TYPE ::= {
    FOO Baz IDENTIFIED BY 2
}

Is there any way to "add" foo-Baz to FooSet (preferably at runtime, so that I don't need to change any of the source ASN.1)?

[p1-bmu]

Hi Ben, these are unfortunately poorly designed ASN.1 modules.

Here, FooThing has a table constraint referring FooSet, which has only 1 value foo-Bar. Then, another value foo-Baz is defined but not added to FooSet, so it's not part of the constraint in FooThing. I believe foo-Baz should be added to FooSet to get everything right at the ASN.1 level ; in your specific case, this would lead to circular import between the 2 modules, and I am not sure it would nicely work. On the other side, having everything correctly defined at the ASN.1 level enables to work as expected with any ASN.1 compiler (being pycrate, asn1c, or any other complete-enough compiler -i.e. commercial ones-).

If you want to patch your objects at runtime, here is how I would do it, working with your example ASN.1 module:

FooModule
DEFINITIONS IMPLICIT TAGS ::=
BEGIN

FOO-TYPE ::= CLASS {
    &id INTEGER UNIQUE,
    &Foo OPTIONAL
} WITH SYNTAX {
    [FOO &Foo] IDENTIFIED BY &id
}

Bar ::= OCTET STRING

foo-Bar FOO-TYPE ::= {
    FOO Bar IDENTIFIED BY 1
}

FooSet FOO-TYPE ::= {
    foo-Bar,
    ...
}

FooThing ::= SEQUENCE {
    id FOO-TYPE.&id({FooSet}),
    fooContent [0] EXPLICIT FOO-TYPE.&Foo({FooSet}{@id})
}

END

BazModule
DEFINITIONS IMPLICIT TAGS ::=
BEGIN

IMPORTS
    FOO-TYPE FROM FooModule;

Baz ::= INTEGER

foo-Baz FOO-TYPE ::= {
    FOO Baz IDENTIFIED BY 2
}

END

After compiling it with $ pycrate_asn1compile.py -i test.asn, one can extend the table constraint of FooThing within Python:

In [1]: from out import *                                                                                                                                 

In [2]: FooModule.FooThing._cont['id']._const_tab # here is the table constraint for FooThing.id
Out[2]: <_tab_FOO-TYPE ([FOO-TYPE] CLASS): ASN1Set(root=[{'Foo': <Foo ([Bar] OCTET STRING)>, 'id': 1}], ext=None)>

In [3]: FooModule.FooThing._cont['fooContent']._const_tab # here is the table constraint for FooThing.fooContent
Out[3]: <_tab_FOO-TYPE ([FOO-TYPE] CLASS): ASN1Set(root=[{'Foo': <Foo ([Bar] OCTET STRING)>, 'id': 1}], ext=None)>

In [4]: FooModule.FooThing._cont['id']._const_tab == FooModule.FooThing._cont['fooContent']._const_tab # they are actually the same object, hence we need to update it just one time, so that the additional value in the constraint will be available for both components of FooThing
Out[4]: True

In [5]: FooModule.FooThing._cont['id']._const_tab._val.root.append( BazModule.foo_Baz._val ) # updating the table constraint with the additional value

In [6]: FooModule.FooThing._cont['id']._const_tab # here it is
Out[6]: <_tab_FOO-TYPE ([FOO-TYPE] CLASS): ASN1Set(root=[{'Foo': <Foo ([Bar] OCTET STRING)>, 'id': 1}, {'Foo': <Foo ([Baz] INTEGER)>, 'id': 2}], ext=None)>

In [7]: FooModule.FooThing._cont['fooContent']._const_tab # for both SEQUENCE components
Out[7]: <_tab_FOO-TYPE ([FOO-TYPE] CLASS): ASN1Set(root=[{'Foo': <Foo ([Bar] OCTET STRING)>, 'id': 1}, {'Foo': <Foo ([Baz] INTEGER)>, 'id': 2}], ext=None)>

Hope this solves your issue.

[benmaddison]

Thanks @p1-bmu That looks like exactly what I need. The ASN.1 design is how everything in CMS-land works, so I have to work around it as best I can! Will let you know if I get it working...

[benmaddison]

@p1-bmu

I have tried out your suggestion. Unless I am missing something, it appears to only half work. The foo-Baz instance is found correctly when setting the value from python data, but isn't found in the lookup table when decoding from DER.

In the below example I am accessing the constraint table slightly differently (to avoid "private" attributes), but I believe it is equivalent to your previous example:

import out

foo_bar_data = {"id": 1,
                "fooContent": ("Bar",
                               {"bar": 100})}

foo_baz_data = {"id": 2,
                "fooContent": ("Baz",
                               {"baz": 100})}

FooThing = out.FooModule.FooThing

foo_const = FooThing.get_internals()["cont"]["id"].get_const()["tab"]
foo_types = (out.BazModule.foo_Baz,)
for t in foo_types:
    foo_const.get_internals()["val"].root.append(t.get_val())

for data in (foo_bar_data, foo_baz_data):
    # set value from python representation
    FooThing.set_val(data)
    print(f"{FooThing.get_val()=}")
    # output:
    # FooThing.get_val()={'id': 1, 'fooContent': ('Bar', {'bar': 100})}
    # FooThing.get_val()={'id': 2, 'fooContent': ('Baz', {'baz': 100})}

    # encode as DER
    der = FooThing.to_der()
    print(f"{der.hex()=}")
    # output:
    # der.hex()='300a020101a0053003020164'
    # der.hex()='300a020102a0053003020164'

    # clear the ASN.1 object
    FooThing.reset_val()
    # decode from DER
    FooThing.from_der(der)
    # dump asn.1 value
    print(f"{FooThing.to_asn1()}")
    # output:
    # {
    #   id 1,
    #   fooContent Bar: {
    #     bar 100
    #   }
    # }
    # OPEN._decode_ber_cont: FooThing.fooContent, unable to retrieve a table-looked up object
    # {
    #   id 2,
    #   fooContent '020164'H
    # }

    # clear the ASN.1 object
    FooThing.reset_val()

Any pointers would be greatly appreciated!

[p1-bmu]

OK, some additional LUT are built when the Python module is initialized. Therefore, after extending your lookup table at runtime, you need to rerun the appropriate initialization routine build_classset_dict on it:

In [1]: from out import * # our example ASN.1 module

In [2]: from pycrate_asn1rt.init import build_classset_dict # the required initialization routine

In [3]: FooModule.FooThing._cont['id']._const_tab._val.root.append( BazModule.foo_Baz._val ) # extending the lookup table

In [4]: build_classset_dict(FooModule.FooThing._cont['fooContent']._const_tab) # re-initializing it

In [5]: FooModule.FooThing.from_der(b'0\n\x02\x01\x02\xa0\x05\x02\x03\x00\xab\xcd') # now it's working :)

In [6]: print(FooModule.FooThing.to_asn1())                                                                                                               
{
  id 2,
  fooContent Baz: 43981
}

Take care also to set the correct values for your content. Here, you should set them like this:

In [7]: foo_bar_data = {"id": 1, 
   ...:                 "fooContent": ("Bar", b"100")} # OCTET STRING 
   ...:  
   ...: foo_baz_data = {"id": 2, 
   ...:                 "fooContent": ("Baz", 100)} # INTEGER                                                                                             

In [8]: FooModule.FooThing.set_val(foo_bar_data)                                                                                                          

In [9]: print(FooModule.FooThing.to_asn1())                                                                                                               
{
  id 1,
  fooContent Bar: '313030'H -- 100 --
}

In [10]: FooModule.FooThing.set_val(foo_baz_data)                                                                                                         

In [11]: print(FooModule.FooThing.to_asn1())                                                                                                              
{
  id 2,
  fooContent Baz: 100
}

[benmaddison]

Thanks @p1-bmu that works perfectly!

For anyone looking for a similar solution in future, the working example is:

import pycrate_asn1rt.init

import out

foo_bar_data = {"id": 1,
                "fooContent": ("Bar",
                               {"bar": 100})}

foo_baz_data = {"id": 2,
                "fooContent": ("Baz",
                               {"baz": 100})}

FooThing = out.FooModule.FooThing

foo_const = FooThing.get_internals()["cont"]["id"].get_const()["tab"]
foo_types = (out.BazModule.foo_Baz,)
for t in foo_types:
    foo_const.get_val().root.append(t.get_val())
pycrate_asn1rt.init.build_classset_dict(foo_const)

for data in (foo_bar_data, foo_baz_data):
    # set value from python representation
    FooThing.set_val(data)
    print(f"{FooThing.get_val()=}")

    # encode as DER
    der = FooThing.to_der()
    print(f"{der.hex()=}")

    # clear the ASN.1 object
    FooThing.reset_val()

    # decode from DER
    FooThing.from_der(der)

    # dump asn.1 value
    print(f"{FooThing.to_asn1()}")

    # clear the ASN.1 object
    FooThing.reset_val()

I have another problem, no that I have solved this! New issue coming up...