we-like-parsers / pegen

PEG parser generator for Python
https://we-like-parsers.github.io/pegen/
MIT License
150 stars 32 forks source link

Bug with zero or more rule #81

Open heckad opened 1 year ago

heckad commented 1 year ago

Gram file

start: stmts

stmts: stmt*

stmt: block

block: "{" stmts "}"

Input

{ {} }

Result

  File "<unknown>", line 1
    { {} }
       ^
SyntaxError: input.txt

The problem in the generated parse rule for the block stmt Code

    @memoize
    def block(self) -> Optional[Any]:
        # block: "{" stmts "}"
        mark = self._mark()
        if (
            (literal := self.expect("{"))
            and
            (stmts := self.stmts())
            and
            (literal_1 := self.expect("}"))
        ):
            return [literal, stmts, literal_1];
        self._reset(mark)
        return None;

(stmts := self.stmts()) shoud be (stmts := self.stmts()) is not None because an empty array is interpreted as false.

0dminnimda commented 1 year ago

stmt* have a value of None if no stmt was found. I haven't checked i, but it think that stmts returns None and block doesn't handle it

An easy, but possible problematic way to solve this is recursively checking expressions until we find single Repeat A possibly more compilated way to solve this is to properly handle this is stmts

I will look into that

0dminnimda commented 1 year ago

Unfortunately here we need a more general indicator of a failure The problem with using None is that if user wants to return it:

start: stmts

stmts: r=stmt* { None if not r else r }

stmt: block

block: "{" stmts "}"
heckad commented 1 year ago

I haven't checked i, but it think that stmts returns None and block doesn't handle it

stmts returns an empty array not None. And that is interpreted as False. None is a good marker.

0dminnimda commented 1 year ago

stmts returns an empty array not None

Yes, initially i was wrong about it

heckad commented 1 year ago

My attempt to solve the problem