Add bytes API.

[toots]

This PR adds an API for tracking position in bytes. It should be opt-in and backward compatible.

Fixes: #139

[toots]

PR is confirmed working with liquidsoap and also confirmed that we were reporting bogus positions before, as we assumed the returned positions were in bytes not code points.

[toots]

@hhugo I think that this one is ready for merge!

[hhugo]

@hhugo I think that this one is ready for merge!

I still think comments are weird. See my comments https://github.com/ocaml-community/sedlex/pull/146/files#r1244060086

[hhugo]

Also see my reply in https://github.com/ocaml-community/sedlex/pull/146#discussion_r1244047393

[toots]

Also see my reply in #146 (comment)

Addressed!

[toots]

@hhugo I think that this one is ready for merge!

I still think comments are weird. See my comments https://github.com/ocaml-community/sedlex/pull/146/files#r1244060086

Addressed!

[hhugo]

LGTM

[hhugo]

Also fixes https://github.com/ocaml-community/sedlex/issues/96

[hackwaly]

So, can we use utf16 positions as well? Vscode lsp currently only support utf16 positions

[toots]

@hackwaly: The lexing_position API returns positions in code-point so utf16 position for utf16 input.