search

jmitchell / tree-sitter-dhall

(WIP) Dhall grammar for Tree-sitter
MIT License
8 stars 1 forks source link

`in` keyword is recognized within an identifier #1

Open jmitchell opened 5 years ago

jmitchell commented 5 years ago

The ignored ./dhall-haskell/dhall-json/examples/travis.dhall parse test fails, in part, because the in in OperatingSystem is parsed as the in keyword of a let expression.

image

When I replace constructors OperatingSystem with OperatingSystem.OSX it fails in the same way (constructors is now deprecated). Using System.OSX, however, allows the subsequent in to parse as expected.

Potential explanations

  1. grammar.js needs to be adjusted using tree-sitter features, like precedence, associativity, token, extras, conflicts, word, and external scanners.
  2. dhall.abnf isn't explicit enough about non-empty whitespace needed around in and possibly other keywords.
  3. dhall.abnf is sufficiently explicit. but abnf-to-tree-sitter didn't translate it correctly to generated_grammar.js, which was the basis for grammar.js.
  4. Edge-case/bug in tree-sitter.
Nadrieril commented 3 years ago

dhall.abnf is indeed not explicit enough: there are some comments in it that explain that simple-label should be tweaked to exclude keywords. I use negative lookahead, but I'd expect a non-PEG parser to handle this at the tokenizer level

jeff-hykin commented 8 months ago

It looks like the bnf file has been updated. Would it be possible to rerun the converter and see if it fixes this?