bzick / tokenizer

Tokenizer (lexer) for golang
MIT License
98 stars 6 forks source link

Added two new flags #10

Closed MrFreemanHR closed 1 year ago

MrFreemanHR commented 1 year ago

Added two flags for parser - for allowing dots ('.') and ats ('@') in keywords

bzick commented 1 year ago

Unfortunately, it's not complete solution because symbols of keywords must be configurable. I added AllowKeywordSymbols. Please try it and give me feedback.