lark-parser / lark

Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.
MIT License
4.88k stars 414 forks source link

Ranges within lists #436

Closed johnny-alvarado closed 5 years ago

johnny-alvarado commented 5 years ago

Currently, I have lists in my grammar. These lists are numbers separated by a comma within parenthesis: (1, 2, 3)

I am trying to include ranges, for example (1-3,8) is equivalent to (1, 2, 3, 8)

Since the minus sign might be confusing, I change it by "&", so my list looks like (1&3,8). For this purpose, I am using the attached grammar and transformers. However, I get an error trying to run the code.

image

My questions are:

  1. What is wrong with my grammar?
  2. Is it possible to change "&" by "-" without causing confusion with the subtract operation?

My code follows...

from lark import Lark, Transformer, v_args
import random

bbx_grammar = """
    ?start: sum
          | TAGNAME "=" sum    -> assign_var
          | TAGNAME ">=" sum    -> increase_var
          | TAGNAME ">" sum    -> increase_var
          | TAGNAME "<=" sum    -> decrease_var
          | TAGNAME "<" sum    -> decrease_var
          | TAGNAME "<>" sum    -> increase_var
          | TAGNAME "IN" list   -> inside_var
          | TAGNAME "NIN" list   -> outside_var
    ?sum: product
        | sum "+" product   -> add
        | sum "-" product   -> sub
        | sum "OR" product  -> or_op
    ?product: atom
        | product "*" atom  -> mul
        | product "/" atom  -> div
        | product "AND" atom -> and_op
    ?list: "(" [atom ("," atom)*] ")"
    ?atom: NUMBER           -> number
         | "-" atom         -> neg
         | TAGNAME          -> var
         | RANGE            -> inside_range
         | "(" sum ")"
    TAGNAME: "{" NAME "}"
    RANGE: NUMBER "&" NUMBER
    %import common.CNAME -> NAME
    %import common.NUMBER
    %import common.WS_INLINE
    %ignore WS_INLINE
"""

@v_args(inline=True)    # Affects the signatures of the methods
class BBXTree(Transformer):
    from operator import add, sub, mul, truediv as div, neg
    number = float

    def __init__(self):
        self.vars = {}

    def assign_var(self, name, value):
        self.vars[name] = value
        return value

    def increase_var(self, name, value):
        self.vars[name] = value + 1
        return value

    def decrease_var(self, name, value):
        self.vars[name] = value - 1
        return value

    def or_op(self, left, right):
        return left or right

    def and_op(self, left, right):
        return left and right  

    def inside_var(self, name, value):
        myList = value.children
        myValue = random.choice(myList)
        self.vars[name] = myValue
        return self.vars[name], myValue

    def outside_var(self, name, value):
        myList = value.children
        myValue = max(myList) + 1
        self.vars[name] = myValue
        return myValue

    def inside_range(self, left, right):
        if left < right:
            myValue = random.randrange(left,right)
        else:
            myValue = left
        self.vars[name] = myValue
        return myValue 

    def var(self, name):
        return self.vars[name]

bbx_parser = Lark(bbx_grammar, parser='lalr', transformer=BBXTree())
bbx = bbx_parser.parse

bbx('{a} IN (1 & 2)')

print(bbx('{a}'))
erezsh commented 5 years ago

RANGE should be a rule (so range), not a terminal. Uppercase vs lowercase matters.

Regarding if a range can be a minus - that depends: Can you explain clearly the difference between range and substractions, in terms of syntax?

Consider using a different range operator, like .. or :

johnny-alvarado commented 5 years ago

I tried with the rule as you suggested and it works. Now, the difference between the range and the substraction is that the range is used only within a list: (1-3,9) is equivalent to (1,2,3,9). Everywhere else it must be a substraction.

erezsh commented 5 years ago

Well, you're gonna have a collision the way you do it now, because a-b can be both. You should put range outside of atom, and as a special member of list.

But I really don't recommend that syntax, what if you want to do subtraction inside lists?

johnny-alvarado commented 5 years ago

Excellent, thank you for your help!