Bad highlighting with emoji `selection_caret`

[korbiniak]

Description

When setting up selection_caret for " " (U+F064) the search highlighting is broken.

Neovim version

NVIM v0.6.1
Build type: Release
LuaJIT 2.1.0-beta3

Operating system and version

Linux 5.17.3-arch1-1

checkhealth telescope

telescope: require("telescope.health").check()
========================================================================
## Checking for required plugins
  - OK: plenary installed.
  - OK: nvim-treesitter installed.

## Checking external dependencies
  - OK: rg: found ripgrep 13.0.0
  - OK: fd: found fd 8.3.2

## ===== Installed extensions =====

## Telescope Extension: `media_files`
  - INFO: No healthcheck provided

## Telescope Extension: `projects`
  - INFO: No healthcheck provided

## Telescope Extension: `ui-select`
  - INFO: No healthcheck provided

Steps to reproduce

1. In the telescope setup, set selection_caret to the emoji " " (U+F064)
2. Try searching anything

Expected behavior

No response

Actual behavior

Broken search highlighting. As you can see, the "r/" is also highlighted. It is not if the carret is for example "> ".

Minimal config

vim.cmd [[set runtimepath=$VIMRUNTIME]]
vim.cmd [[set packpath=/tmp/nvim/site]]
local package_root = '/tmp/nvim/site/pack'
local install_path = package_root .. '/packer/start/packer.nvim'
local function load_plugins()
  require('packer').startup {
    {
      'wbthomason/packer.nvim',
      {
        'nvim-telescope/telescope.nvim',
        requires = {
          'nvim-lua/plenary.nvim',
          { 'nvim-telescope/telescope-fzf-native.nvim', run = 'make' },
        },
      },
      -- ADD PLUGINS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
    },
    config = {
      package_root = package_root,
      compile_path = install_path .. '/plugin/packer_compiled.lua',
      display = { non_interactive = true },
    },
  }
end
_G.load_config = function()
  require('telescope').setup {
    defaults = {
      selection_caret = " ",
    }
  }
  require('telescope').load_extension('fzf')
  -- ADD INIT.LUA SETTINGS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
end
if vim.fn.isdirectory(install_path) == 0 then
  print("Installing Telescope and dependencies.")
  vim.fn.system { 'git', 'clone', '--depth=1', 'https://github.com/wbthomason/packer.nvim', install_path }
end
load_plugins()
require('packer').sync()
vim.cmd [[autocmd User PackerComplete ++once echo "Ready!" | lua load_config()]]

[Conni2461]

Do you usually use fzf-native? Because it calculates the highlight positions (i can reproduce it with fzf-native) But it would also be good if we check the internal sorters as well as fzy-native

[korbiniak]

I don't, I used it here because it was in the template for mre.

[joeveiga]

Also setting the selection_caret to anything more than 2 chars causes the same problem. Here I set it to "> " (2 ws):

Testing on fzf itself I get:

$ find . | fzf --pointer=""
invalid character in pointer

And with more than 2 characters:

$ find . | fzf --pointer=">  "
pointer display width should be up to 2

So, maybe not supported?

[joeveiga]

I suspect the issue is related to some weird byte indexing/encoding shenanigans in this call:

https://github.com/nvim-telescope/telescope.nvim/blob/e2a77a54a35642dd95310effe2bf4e36fff3af26/lua/telescope/pickers.lua#L293

After some vim.inspecting of the parameters passed into that function everything seems fine up to that point. I tried playing with vim.str_byteindex(..) but no luck.

[fitrh]

Setting it to an empty string (selection_caret = "") also breaks highlight matching

Do you usually use fzf-native? Because it calculates the highlight positions (i can reproduce it with fzf-native) But it would also be good if we check the internal sorters as well as fzy-native

I can reproduce it with zf-native (natecraddock/telescope-zf-native.nvim)