Add support for translating between character offsets and code point indices

[wooorm]

Initial checklist

• [X] I read the support docs
• [X] I read the contributing guide
• [X] I agree to follow the code of conduct
• [X] I searched issues and couldn’t find anything (or linked relevant results below)

Problem

Related: https://github.com/syntax-tree/unist/issues/99.

remark-lint where a user provides a max size of say 80, and they don’t mean the way that JS stores strings. To check if things are (not) allowed at that point, a lint tool needs to be aware of that difference, and be able to translate between the two.

Solution

Two new methods: toCodePointIndex(offset: number): number / fromCodePointIndex(index: number): number

Alternatives

Different utility.

[remcohaszing]

remark-lint where a user provides a max size of say 80

I think you mean a max line length by this? In that case, I think you just need to calculate the code point index on the current line, not of the whole file.

[wooorm]

Yes, it would possible in that case to only look at single lines.

This package can be passed single lines; if that remark-lint rule or other potential use-cases want to implement shortcuts like that, they can. The conversion still needs to be implemented somewhere.

But several remark-lint rules already use this package, to convert between offsets and points. As the entire file is already indexed once, and that rule already looks at every single line, I’m not sure how much it would help to look at single lines.