PyParsing issue (and work-around)

[vanrein]

Problem: Using asn1ate 0.6.0 as packaged for Python 3.9, we discovered a problem with PyParsing 3.0.6; this caused exceptions when using OPTIONAL or DEFAULT x annotations,

  File ".../asn1ate-0.6.0-py3.7.egg/asn1ate/sema.py", line 474, in <listcomp>
    for token in component_tokens]
  File ".../asn1ate-0.6.0-py3.7.egg/asn1ate/sema.py", line 994, in _create_sema_node
    raise Exception('Unknown token type: %s' % token.ty)
Exception: Unknown token type: ComponentTypeOptional

Workaround: We worked around it by demanding an older version, namely 3.0.0, of PyParsing.

Question: It is not quite clear if this is only due to PyParsing; there may be something wrong with asn1ate that gets triggered. This most likely is in sema.py function _create_sema_node() which has

    elif token.ty == 'ComponentType':
        return ComponentType(token.elements)

but no cases for ComponentTypeOptional, ComponentTypeDefault, ComponentTypeComponentsOf. When I added such cases under PyParsing 3.0.6, it ran into a problem using the elements array, which I could not furhter assign to my thinking or that in the sema.py module.

[kimgr]

Thank you! Funny enough, I experimented with asn1ate just last week and ran into the same problem. And settled on the same workaround 🙂. But it sounds like there's a deeper problem that should be addressed, so I'll try to get to the bottom of it and see if there's a solution that works with both old and new pyparsing.

[kimgr]

I filed this: https://github.com/pyparsing/pyparsing/issues/345. Reverting the commit https://github.com/pyparsing/pyparsing/commit/9987004c94ccf7d9b6b3adbcf06d05d2ff197737 makes asn1ate's test suite work again, so I'm inclined to think it's an isolated pyparsing behavior that triggers it.

[vanrein]

Thanks for teasing this out, Kim. We will await pyparsing 3.0.7 then.

[kimgr]

Thanks, I just pushed a stricter setup.py excluding the known-broken versions from install dependencies. It turns out 3.0.4 also works, so you can install that as a workaround.

Closing.