fix tokenizer to parse regexp modifier as part of the regular expression

[njeisecke]

This

• allows to use modifiers like 'u' (for unicode support) to be used.
• makes the Rfc822UTF8.peg.inc example working again.

This change might break regexp rules not followd by a whitespace or a line-end.