The Gumbo tokenizer assumed that the start of the first token is at the
beginning of the input. This is not the case if the input starts with a
UTF-8 byte-order mark.
This change removes that assumption by asking the iterator itself for
the pointer to the start of the token.
The Gumbo tokenizer assumed that the start of the first token is at the beginning of the input. This is not the case if the input starts with a UTF-8 byte-order mark.
This change removes that assumption by asking the iterator itself for the pointer to the start of the token.
Fixes #157