Use lookup tables instead of linear search with `memchr`

[lopopolo]

Several places use memchr with a byteset. This commit refactors these code paths to construct a lookup table from u8 -> bool, where an index is set to true if the byte is present in the given slice.

This change removes linear scans that occur in loops, which changes these functions runtime complexity from O(m * n) to O(m + n).

[gwenn]

I guess break_chars_byteset can be evaluated once / statically. Something like https://github.com/sqlite/sqlite/blob/master/src/tokenize.c#L61-L80 but for DEFAULT_BREAK_CHARS and DOUBLE_QUOTES_SPECIAL_CHARS.

[lopopolo]

@gwenn I think so too but I wasn't sure if that change would be accepted since these functions are public APIs that take the byteset as a slice.

I'm not sure how you'd like to make the API breaks.

Do you want to merge this as is or maybe push a PR to my fork?

[gwenn]

See #676

[lopopolo]

Thanks @gwenn!