Accept a list of match objects

no-context / moo

Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.

BSD 3-Clause "New" or "Revised" License

821 stars 65 forks source link

Accept a list of match objects #62

Closed tjvr closed 7 years ago

tjvr commented 7 years ago

This allows writing rules in the object style, where the rules have the same tokenType but different properties (e.g. lineBreaks, or state manipulation).

e.g.

  STRING: [
    {match: /"""[^]*?"""/, lineBreaks: true},
    {match: /"(?:\\["\\]|[^"\\\n])*?"/},
    {match: /'(?:\\['\\]|[^'\\\n])*?'/},
  ],

nathan commented 7 years ago

Do we want it to accept mixed arrays like [/regexp/, 'string', {match: /something/}]? If so, we should test for that.