After evaluating things, I'm not sure if full unicode support is required. I looked at other fuzzy finders, and none that I checked (fzf, fzy, sk) fully support unicode in the way I was imagining to support for zf. I want to prioritize speed and simplicity first. So I want to simplify the unicode implementation. Here is the plan:
[x] normalization of input lines and query
this only has a minimal impact on performance, and will make matching unicode mostly perfect
[x] fixing all code that uses the length of a string as the display width
[x] query editing
This will actually be better than any other I tested (handling graphemes in the query editor line), but also keep zf as optimized as possible.
I am open to changing things in the future, but for now I think this is enough.
After evaluating things, I'm not sure if full unicode support is required. I looked at other fuzzy finders, and none that I checked (fzf, fzy, sk) fully support unicode in the way I was imagining to support for zf. I want to prioritize speed and simplicity first. So I want to simplify the unicode implementation. Here is the plan:
This will actually be better than any other I tested (handling graphemes in the query editor line), but also keep zf as optimized as possible.
I am open to changing things in the future, but for now I think this is enough.