ohmjs / ohm

A library and language for building parsers, interpreters, compilers, etc.
MIT License
4.97k stars 219 forks source link

Indentation-sensitive parsing: dedent not working? #467

Open poke1024 opened 6 months ago

poke1024 commented 6 months ago

Here is an example modified from https://github.com/ohmjs/ohm/blob/main/examples/indentation-sensitive/index.js, that I would expect to work. Note that Block has been changed to takeExpr+ instead of a single Expr.

import {grammar, ExperimentalIndentationSensitive as IndentationSensitive} from 'ohm-js';

const g = grammar(
    String.raw`
G <: IndentationSensitive {
  IfExpr = "if" Expr ":" Block
  Block = indent Expr+ dedent
  Expr = IfExpr
       | "True"
       | "False"
       | number

  number = digit+
}
`,
    {IndentationSensitive}
);

const examples = [
`
if True:
  3
`,
`
if True:
  if True:
    3
`,
`
if True:
  if True:
    3
  if True:
    3
`
];

for (const s of examples) {
    const r = g.match(s.trim());
    if (!r.succeeded()) {
        console.log("failed on:", s);
        console.log(r.message);
    }
}

The first two examples in examples work, however the third (rather trivial) example gives this error:

Line 4, col 3:
  3 |     3
> 4 |   if True:
        ^
  5 |     3
Expected a dedent

Similarly, any actual dedent onto a previous level seems to issue this problem.

poke1024 commented 6 months ago

By the way, running findIndentation on the code gives very reasonable results, namely { '11': 1, '24': 1, '28': -1, '41': 1, '42': -2 }, which are exactly the expected indents and dedents.

pdubroy commented 6 months ago

Interesting...thanks for reporting. Might be a problem with space skipping, but I'll investigate.

simonwo commented 6 months ago

FYI this looks superficially similar to my issue in #465.