Version 1.2.1 of @lezer/generator tokenizes differently

[viktor-berglund-mw]

I have a grammar like this:

@top Program { expression }

@skip { Blank }

expression { (Number | Identifier)+ }

@tokens {
  Number { $[0-9]+ }
  Identifier { $[0-9a-z]+ }
  Blank { ' ' }

  @precedence { Number, Identifier }
}

In version 1.2.0. the string 1a is tokenizes as: Program(Identifier). In version 1.2.1, 1a is tokenized as: Program(Number,Identifier)

[marijnh]

Yes, there was a bug in versions 1.2.0 and below, causing @precedence to not be respected in cases like this. The current behavior is the intended behavior.

[viktor-berglund-mw]

Aha thanks for the response!