search

pycrate-org / pycrate

A Python library to ease the development of encoders and decoders for various protocols and file formats, especially telecom ones. Provides an ASN.1 compiler and a CSN.1 runtime.
https://github.com/pycrate-org/pycrate
GNU Lesser General Public License v2.1
31 stars 9 forks source link

WITH COMPONENTS issue: AssertionError or compiler error during asn1 compilation of IEEE1609 #8

Open majorcs opened 4 months ago

majorcs commented 4 months ago

I'm trying to compile the IEEE 1609.2.1 ASN with pycrate. The ASN files could be downloaded from here: IEEE Download

The compilation cycle fails with an assertion. Could it be a bug or it is the limitation of pycrate?

$ pycrate_asn1compile.py -i asn1/1609.2.1/Ieee1609Dot2Asn/ -i asn1/1609.2.1/ -o python/1609_2
[proc] [asn1/1609.2.1/Ieee1609Dot2BaseTypes.asn] module Ieee1609Dot2BaseTypes (oid: [1, 3, 111, 2, 1609, 2, 1, 2, 2, 3]): 74 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1AcaEeInterface.asn] module Ieee1609Dot2Dot1AcaEeInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 1, 2, 3]): 3 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1AcaMaInterface.asn] module Ieee1609Dot2Dot1AcaMaInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 3, 2, 1]): 2 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Peer2Peer.asn] module Ieee1609Dot2Peer2Peer (oid: [1, 3, 111, 2, 1609, 2, 2, 1, 2, 2]): 3 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2CrlSsp.asn] module Ieee1609Dot2CrlSsp (oid: [1, 3, 111, 2, 1609, 2, 3, 3, 2, 1]): 4 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1EeMaInterface.asn] module Ieee1609Dot2Dot1EeMaInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 10, 2, 1]): 2 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Crl.asn] module Ieee1609Dot2Crl (oid: [1, 3, 111, 2, 1609, 2, 3, 3, 1]): 3 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1LaMaInterface.asn] module Ieee1609Dot2Dot1LaMaInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 12, 2, 1]): 2 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1MaRaInterface.asn] module Ieee1609Dot2Dot1MaRaInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 14, 2, 1]): 2 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1Acpc.asn] module Ieee1609Dot2Dot1Acpc (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 18, 3, 1]): 11 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2CrlBaseTypes.asn] module Ieee1609Dot2CrlBaseTypes (oid: [1, 3, 111, 2, 1609, 2, 3, 2, 3, 1]): 17 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1Protocol.asn] module Ieee1609Dot2Dot1Protocol (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 17, 3, 1]): 62 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1CamRaInterface.asn] module Ieee1609Dot2Dot1CamRaInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 19, 2, 2]): 5 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1AcaRaInterface.asn] module Ieee1609Dot2Dot1AcaRaInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 4, 3, 1]): 9 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1CertManagement.asn] module Ieee1609Dot2Dot1CertManagement (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 7, 3, 1]): 22 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2HeaderInfoExtensionBase.asn] module Ieee1609Dot2HeaderInfoExtensionBase (oid: [1, 3, 111, 2, 1609, 2, 1, 3, 2, 1]): 7 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2.asn] module Ieee1609Dot2 (oid: [1, 3, 111, 2, 1609, 2, 1, 1, 2, 5]): 42 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1AcaLaInterface.asn] module Ieee1609Dot2Dot1AcaLaInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 2, 2, 1]): 2 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1EeRaInterface.asn] module Ieee1609Dot2Dot1EeRaInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 11, 3, 1]): 9 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1EcaEeInterface.asn] module Ieee1609Dot2Dot1EcaEeInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 9, 3, 1]): 4 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Dot1LaRaInterface.asn] module Ieee1609Dot2Dot1LaRaInterface (oid: [1, 3, 111, 2, 1609, 2, 255, 1, 1, 13, 2, 1]): 2 ASN.1 assignments found
--- compilation cycle ---
INF: Ieee1609Dot2Dot1Protocol.ScmsPdu.content, unprocessed CONSTRAINED BY constraint
Traceback (most recent call last):
  File "/home/major/Downloads/asn1-2024-draft/pycrate/bin/pycrate_asn1compile.py", line 264, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/major/Downloads/asn1-2024-draft/pycrate/bin/pycrate_asn1compile.py", line 215, in main
    compile_text(txt, **ckw)
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnproc.py", line 254, in compile_text
    compile_modules(remain)
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnproc.py", line 942, in compile_modules
    ObjNew = asnobj_compile(Obj)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnproc.py", line 876, in asnobj_compile
    text = Obj.parse_def(Obj._text_def)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 1716, in parse_def
    rest = self._parse_const(text)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 3976, in _parse_const
    self._parse_const_withcomps(const)
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 4284, in _parse_const_withcomps
    self.__parse_const_withcomps_comp(rc, const_comp)
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 4376, in __parse_const_withcomps_comp
    comp = Comp._parse_const(comp)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 3964, in _parse_const
    self._parse_const_val(const)
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 4033, in _parse_const_val
    self.parse_set(const['text'])
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 5844, in parse_set
    rest = self._parse_set_comp(rv, val, dom)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 6040, in _parse_set_comp
    text = ObjProxy.parse_def(text)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 1716, in parse_def
    rest = self._parse_const(text)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 3970, in _parse_const
    self._parse_const_size(const)
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 4148, in _parse_const_size
    ObjProxy.parse_set(text)
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 5838, in parse_set
    self.__parse_set_comp_path_config(val, dom)
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 5910, in __parse_set_comp_path_config
    _path_trunc(2)
  File "/home/major/Downloads/asn1-2024-draft/pycrate/lib/python3.11/site-packages/pycrate_asn1c/asnobj.py", line 191, in _path_trunc
    assert( len(GLOBAL.COMP['NS']['path'][-1]) >= depth )
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
mitshell commented 4 months ago

In my recollection, these specs rely heavily on WITH COMPONENTS, which is not perfectly supported within pycrate. This will need some more investigation...

mitshell commented 4 months ago

When trying to reproduce, I get a different error: 

pycrate_asn1c.err.ASN1ProcTextErr: Ieee1609Dot2Data-Unsecured: missing mandatory components in WITH COMPONENTS

That means pycrate is not happy when the WITH COMPONENTS sub-select CHOICE components, which seems legit according to ASN.1. This confirms this overall pattern is not properly implemented in pycrate. The code lies here: https://github.com/pycrate-org/pycrate/blob/4d58c2bb5f7d16bdb9b7c110f829aff856be49c0/pycrate_asn1c/asnobj.py#L4258

This will need a dive into the code for proper support...

majorcs commented 4 months ago

I think I made some changes to the ASN1 and didn't revert it fully, so this is why I got a different error message.

I also tried to compile only the 1609.2 part which is the 1609.2.1 built on 

$ pycrate_asn1compile.py -i asn1/1609.2.1/Ieee1609Dot2Asn/ -o python/1609_2
[proc] [asn1/1609.2.1/Ieee1609Dot2Asn/Ieee1609Dot2BaseTypes.asn] module Ieee1609Dot2BaseTypes (oid: [1, 3, 111, 2, 1609, 2, 1, 2, 2, 3]): 74 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Asn/Ieee1609Dot2Peer2Peer.asn] module Ieee1609Dot2Peer2Peer (oid: [1, 3, 111, 2, 1609, 2, 2, 1, 2, 2]): 3 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Asn/Ieee1609Dot2CrlSsp.asn] module Ieee1609Dot2CrlSsp (oid: [1, 3, 111, 2, 1609, 2, 3, 3, 2, 1]): 4 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Asn/Ieee1609Dot2Crl.asn] module Ieee1609Dot2Crl (oid: [1, 3, 111, 2, 1609, 2, 3, 3, 1]): 3 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Asn/Ieee1609Dot2CrlBaseTypes.asn] module Ieee1609Dot2CrlBaseTypes (oid: [1, 3, 111, 2, 1609, 2, 3, 2, 3, 1]): 17 ASN.1 assignments found
[proc] [asn1/1609.2.1/Ieee1609Dot2Asn/Ieee1609Dot2HeaderInfoExtensionBase.asn] module Ieee1609Dot2HeaderInfoExtensionBase (oid: [1, 3, 111, 2, 1609, 2, 1, 3, 2, 1]): 7 ASN.1 assignments found
Traceback (most recent call last):
  File "/usr/bin/pycrate_asn1compile.py", line 264, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/bin/pycrate_asn1compile.py", line 215, in main
    compile_text(txt, **ckw)
  File "/home/major/.local/lib/python3.11/site-packages/pycrate_asn1c/asnproc.py", line 234, in compile_text
    mod_names.extend( _compile_text_pass(t, with_order, **kwargs) )
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/major/.local/lib/python3.11/site-packages/pycrate_asn1c/asnproc.py", line 389, in _compile_text_pass
    raise(ASN1ProcTextErr('[proc]{0} module {1}: duplicate object, {2}'\
pycrate_asn1c.err.ASN1ProcTextErr: [proc] [asn1/1609.2.1/Ieee1609Dot2Asn/Ieee1609Dot2.asn] module Ieee1609Dot2: duplicate object, IEEE1609DOT2-HEADERINFO-CONTRIBUTED-EXTENSION
mitshell commented 3 months ago

I don't have time currently to dive in the ASN.1 compiler code, so I'll keep this issue on hold unless someone volunteers to give it a try.