no-context / moo

Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.
BSD 3-Clause "New" or "Revised" License
821 stars 65 forks source link

Sort literals globally #59

Closed tjvr closed 7 years ago

tjvr commented 7 years ago

Currently, literals in earlier rules will take precedence over later rules, even if they're a prefix.

  test('prioritises literals correctly', () => {
    let lexer = moo.compile({
      one: 'moo',
      two: 'moomintroll',
    })
    lexer.reset('moomintroll')
    expect(lexer.next()).toMatchObject({type: 'two'}) // fails
  })

This is probably not what we want. We're already special-casing literals; it might not unreasonable to special-case 'em further? Then again, magic is bad. Argh.

nathan commented 7 years ago

This seems like the correct behavior. If you replaced them with /moo/ and /moomintroll/ you'd get the same result, which is good.

tjvr commented 7 years ago

I take it “this” refers to the existing behaviour? So you’d vote to close this?

nathan commented 7 years ago

Correct.

tjvr commented 7 years ago

Closing; "works as intended".

tjvr commented 7 years ago

084b235 adds a test for the current behaviour.