tree-sitter / tree-sitter-cli

CLI tool for creating and testing tree-sitter parsers
MIT License
45 stars 15 forks source link

Non greedy regular expressions aren't working. #56

Open jonhue opened 5 years ago

jonhue commented 5 years ago

I have a grammar that uses this regex to match comments (it's in the extras array):

comment: $ => /((\/\/[^\r\n]*)|(\/\*(.|\s)*?\*\/))/

This fails on the following test:

=====================================
Comments
=====================================

1 // C$mment
1// C$mment
1/* C$mment
C$mment

C$mment*/
1

---

(file
  (integer) (comment)
  (integer) (comment)
  (integer) (comment)
  (integer))

However, when I make the regex greedy (/((\/\/[^\r\n]*)|(\/\*(.|\s)*\*\/))/), the test is successful. The regex has to be non greedy for cases like this:

1 // C$mment
1// C$mment
1/* C$mment
C$mment

C$mment*/ + 1
/* C$mment */
1
maxbrunsfeld commented 5 years ago

Yeah, non-greedy regexes are not supported.

It looks like you're trying to match C-style comments. If so, have you tried this regex?