Using Lark Standalone Parser with ansible

[x23n5902y]

I made Lark Standalone Parser according to the instructions to write a module that won't have a dependency on the Lark package

python -m lark.tools.standalone grammar.lark > my _parser .py

When I try to import it in ansible module

from ansible.module_utils.my_parser.my_parser import Lark_StandAlone, Transformer
parser = Lark_StandAlone(transformer=TreeToJson())

I get the error

msg: Could not find imported module support code for my_parser. Looked for either Token.py or lexer.py

ansible debug

Using module_utils file /home/polemarch/1/parser/module_utils/my_parser/__init__.py
Using module_utils file /opt/polemarch/lib64/python3.6/site-packages/ansible/module_utils/common/text/converters.py
Using module_utils file /opt/polemarch/lib64/python3.6/site-packages/ansible/module_utils/common/text/__init__.py
Using module_utils file /opt/polemarch/lib64/python3.6/site-packages/ansible/module_utils/common/__init__.py
Using module_utils file /home/polemarch/1/parser/module_utils/my_parser/my_parser.py
Using module_utils file /opt/polemarch/lib64/python3.6/site-packages/ansible/module_utils/basic.py
Using module_utils file /opt/polemarch/lib64/python3.6/site-packages/ansible/module_utils/common/_collections_compat.py
Using module_utils file /opt/polemarch/lib64/python3.6/site-packages/ansible/module_utils/six/__init__.py
Using module_utils file /opt/polemarch/lib64/python3.6/site-packages/ansible/module_utils/_text.py

[MegaIng]

This is something to report to ansible, not to us. The standalone file is standalone hand has no "support code".

[x23n5902y]

This is an import error from a standalone file. There are no problems with importing from the Lark module. I don't think this is ansible specific problem. I think using a standalone file is very useful with ansible as it gets rid of dependency resolution

[MegaIng]

It's 100% is a problem with ansible. The error message does not come from lark. If you can generate an error without ansible, then we could look into this.

[erezsh]

There are no problems with importing from the Lark module

@x23n5902y This looks like pathing issues. Is your standalone in the same directory as the module that uses Lark directly? Are you calling them with the same working directory? If not, that might be why ansible is failling.

[x23n5902y]

I follow instructions https://docs.ansible.com/ansible/latest/dev_guide/developing_module_utilities.html

To reduce the maintenance burden in a collection or in local modules, you can extract duplicated code into one or more module utilities and import them into your modules. For example, if you have your own custom modules that import a my_shared_code library, you can place that into a ./module_utils/my_shared_code.py file like this:

from ansible.module_utils.my_shared_code import MySharedCodeClient

The problem occurs only with the standalone parser, which is obtained by

python -m lark.tools.standalone grammar.lark > my_parser.py

command. I can successfully import my code from other files in the same directory

[MegaIng]

I assume it's something like ansible trying to be smart and seeing unresolved references/imports in the standalone file, despite those never being used and therefore not causing problems during execution. The standalone parser works, ansible is generating errors for some reason.

[x23n5902y]

let's see what the ansible guys have to say

[x23n5902y]

The problem was in my old version of ansible

Looking at that standalone.py, it contains all of the necessary code, but extracts portions into separate files at runtime. This means that the files don't exist and the imports cannot be walked at build time.

In 2.9.20 we added support for "optional" imports that will not trip up the walker code. See https://github.com/ansible/ansible/pull/73916