Regexp tokenizer/scanner in Go.
The scanner takes in a list of token definitions and string input. it is general and easy to use.
a token is a pair of constant and regexp string.