Using a model/listener with a lexer grammar

[ZekReshi]

I am playing around with the Java grammar and models/listeners, where the lexer grammar contains the following rules:

IntegerLiteral:
    DecimalIntegerLiteral
    | HexIntegerLiteral
    | OctalIntegerLiteral
    | BinaryIntegerLiteral
;

fragment DecimalIntegerLiteral: DecimalNumeral IntegerTypeSuffix?;

fragment HexIntegerLiteral: HexNumeral IntegerTypeSuffix?;

fragment OctalIntegerLiteral: OctalNumeral IntegerTypeSuffix?;

fragment BinaryIntegerLiteral: BinaryNumeral IntegerTypeSuffix?;

Suppose I want the quantifier in DecimalIntegerLiteral to be treated differently from the other quantifiers, how is this possible? The model's quantify method only receives the parent node (the node representing IntegerLiteral) and idx is 0 for all quantifiers as they're the first in the fragment.

[renatahodovan]

First of all, you need to checkout the latest master of Grammarinator, since it contains a fix which is needed to handle sub-lexer rules properly.

From this point, you simply need to define a custom DispatchingModel that overrides the quantifier handling of the chosen rule. In you case, you need something like this:

java_model.py:

import random

from grammarinator.runtime import DispatchingModel

class JavaModel(DispatchingModel):

    def quantify_DecimalIntegerLiteral(self, node, idx, cnt, start, stop):
        print('Custom decision for the quantifiers of DecimalIntegerLiteral')
        if idx == 0:  # There is only one quantifier in this rule, hence this condition would not be necessarily.
            if cnt < stop:  # Do not exceed the maximum allowed quantified items.
                # Implement your decision mechanism here and return True or False accordingly.
                return random.choice([True, False])
            return False

Finally, register this modul as a model of grammarinator-generate (and don't forget to add the directory of the model to PYTHONPATH with the --sys-path argument):

grammarinator-generate JavaGenerator.JavaGenerator --stdout \
    --model JavaModel.JavaModel -r IntegerLiteral \
    -j=1 --sys-path <path/to/java_model.py>