[Bug]: Movement stops at non-English accented letters

[maxigaz]

Bug Description

In normal mode, while the cursor is at the beginning of a word that has at least one accented letter in it, pressing w will jump to the character following that letter instead of to the next word.

Reproduction & Sample Text

before: |tükörfúrógép teszt after: tü|körfúrógép teszt expected: tükörfúrógép |teszt

In comparison, if the word has no accented letters in it (as in tukorfurogep teszt), pressing w works as expected.

neovim version

v0.9.0

Special Settings

No.

Make sure you have done the following

• [X] In case I have an issue with using the motions in operater-pending mode (cw, de, …), I read the notes on operator-pending mode in the README.
• [X] I have updated to the latest version of the plugin.

[maxigaz]

Looking at this part of the code tells me nvim-spider relies on regular Lua string patterns such as %u for uppercase letters.

To my knowledge, the reason accented letters are ignored in such patterns is because Neovim relies on LuaJIT, which is based on Lua 5.1, which doesn't support the UTF8 character set (unlike later versions of Lua).

One option could be to rewrite that part of the code in order to use the external library utf8, and distribute the library with the plugin. I'm not very familiar with plugin development at this point, so I'm not sure if this would be the 'right' way of doing it though.

[chrisgrieser]

good find. yeah, I assume that adding some library like that would be necessary. however, I am a bit short on time for the next couple of weeks, so feel free to submit a PR if you figure it out?

[JarKz]

Hello and welcome!

I tried to make it, found some libraries that do it, but need to understand some things. First, I found the useful C library in github, which can be compiled by luarocks: starwing/luautf8. But, as you know, it's not simple to add non-plugin for neovim lua library. That's make some complexity to install but I also find these things:

1. packer.nvim already have some functionality that installs non-plugin for neovim lua library (keyword use_rocks);
2. if user don't use the packer.nvim then he can install plugin theHamsta/nvim_rocks that also do. In the second case, need put some code in dependencies. For example, in lazy.nvim:

   return {
   "chrisgrieser/nvim-spider",
   dependencies = {
       "starwing/luautf8",
       {
           "theHamsta/nvim_rocks",
           event = "VeryLazy",
           build = "pip3 install --user hererocks && python3 -mhererocks . -j2.1.0-beta3 -r3.0.0 && cp nvim_rocks.lua lua",
           config = function()
               local rocks = require("nvim_rocks")
               rocks.ensure_installed("luautf8")
           end,
       },
   },
   }

Finally, I've writed code for me and I've found nuances. col indexing will be deprecated in the method getNextPosition and firstMatchAfter because UTF-8 contains bytes that have different widths and need use offset instead of this. As you see, need rewrite some code. I don't know how you feel about this, so I'd like to know what you think about it, @chrisgrieser?

[backdround]

I've reimplemented the plugin and fixed the issue as well. neowords.nvim

[chrisgrieser]

added in the latest release, see readme for adding the required luarocks dependency for it