Use async/await in grammar executed code

[jmaister]

I am writing a grammar to interact with a asynchronous service. So I would like to use await in the executed code.

...
| TYPE SELECTOR expr
    {
        await yy.type($2, $3);
    }
| ...

As the code block is not an async function, the await can't be used there.

Would be any way to call async functions and wait until its completion to continue to the next grammar element ?

[jmaister]

I analysed the generated code of the parser and started adding async/await to the methods that executed the code blocks.

This is the diff file between the two parsers. It starts adding async to the parse function and so on:

-parse: function parse(input) {
+parse: async function parse(input) {

async_diff.txt

How these changes in the parser could be translated into the parser generator ? Could it be possible ?

(I also have the full files if needed.)

[GerHobbelt]

After a quick read of the above, the basic answer would be "yes". As you have already shown yourself given async_diff.txt 👍

However... the TL;DR version

However, implementing it in the jison code generator is a bit of work :wink:. With the current codebase I wouldn't try to add that feature as the current code generator is not powerful enough to accomplish the required kernel code processing/rewriting without turning that part of jison internals into a real horror.

The next major release of jison SHOULD be babel-transform-based, which would make producing both these async parsers and the vanilla (sync) variants a far more manageable feature for the jison generator.

However... the long-winded yadayada version

However, implementing it in the jison code generator is a bit of work :wink:. With the current codebase I wouldn't try to add that feature as the current code generator uses a relatively simple code processing approach to produce a parser (and lexer) for a given grammar: the 'generic parser kernel' is edited using regex filter expressions which are driven by an analysis stage which inspects your grammar and action code blocks to see which jison language features (e.g. location tracking via yylloc, etc. or absence vs. presence of error detection/recovery grammar rule productions) are actually used and and removing those chunks of parser kernel code which are unused and would otherwise only produce a larger and slower parser engine for your given grammar and actions.

I am looking into migrating this 'code production' process to a babel-based approach where the generic parser and lexer kernels plus all user-defined action code blocks is turned into a proper AST, then apply AST rewrites (done as babel plugins) driven by those feature detect flags mentioned above and thus produce a suitably stripped/optimized parser+lexer for your grammar spec.

I.e. replace the somewhat-complicated-by-now regex-based somewhat-brittle code rewriting activity with a babel+AST-rewrite-plugins based transform activity. It wouldn't reduce the complexity from my POV but I expect it will ultimately give us a more manageable jison codebase which will only be 'brittle' in the sense that it'll be tightly dependent on babel architecture (babel lexer, parser and plugin architecture).

Once jison generates parsers via such an AST rewriting process I believe it would be much more doable to make this idea/request a reality by defining a generic async parser kernel and have the AST rewrite process strip off the await and async code chunks for everyone who just wants a fast 'regular vanilla jison parser'.

Anyway, that's my blurb on this for now. the babel-based approach is not yet a working reality and I think that should be done first, otherwise this would make the existing jison too messy, internally. Which is kinda important to me. :wink:

[jmaister]

Thanks for your answer.

Finally I decided to use a different approach to have something working and avoid using the await/async.