Cleanup the scanner - Githubissues

yitzchak / tree-sitter-latex

LaTeX grammar for tree-sitter

MIT License

11 stars 5 forks source link

Closed yitzchak closed 5 years ago

yitzchak commented 5 years ago

Rename TokenType to SymbolType
Rename CategoryDescription to SymbolDescription and scan_category to scan_symbol
Use a bitset for category flags in SymbolDescription
Use scan_symbol for _TOKEN_END and _NON_LETTER_OR_OTHER