Add proper unicode support in the lexer

[TethysSvensson]

Also consider if we want to support surrogate pairs like in the upstream parser. If not, we need to document it somewhere.