Cannot create custom surround pairs with multibyte-characters

[Mango0x45]

Checklist

• [X] Have you tried updating the plugin to the latest version?
• [X] Have you checked the Breaking Changes issue?
• [X] Have you read through :h nvim-surround to see if there might be any relevant information there?

Neovim Version

v0.9.4

Plugin Version

Tagged (Stable)

Minimal Configuration

require('nvim-surround').setup {
    surrounds = {
        ['’'] = {
            add = { '‘', '’' },
        }
    }
}

Sample Buffer

fo*o

(* is the cursor)

Keystroke Sequence

ysiw’

Expected behavior

‘foo’

Actual behavior

’foo’

Additional context

In case the GitHub font makes it hard to see, I am trying to create a custom surround pair using unicode curly quotes, which I have on my keyboard. I expect to be able to surround things with an opening curly quote and a closing curly quote, but the plugin completely ignores my configuration and just uses the closing quote on both ends. If I change the config to be ['X'] = { add = … } } where X is any regular ASCII character, this issue stops.

[Mango0x45]

Update: Did a little testing, the issue seems to not be for all multi-byte characters? Doing ['ß'] = … works just fine, as does ['ẞ'] = ….

[kylechui]

The default behavior is to duplicate the character on both sides, which seems to be what you are experiencing. Maybe try <C-v>[your funny quote character] to see if nvim is even registering it properly? The fact that other multi-byte characters work makes this a bit harder to debug...

[Mango0x45]

Neovim does indeed register it properly.

[Mango0x45]

Ok so it seems to also not work with the multi-byte ‘…’ character. Could it be an issue with punctuation? Lol

[kylechui]

I'm not sure; I don't have any special code to handle retrieving the character. Under the hood, things just use vim.fn.getcharstr.

[Mango0x45]

Well I’m playing around with :lua print(vim.fn.getcharstr()) and seem to have absolutely no problems with inputting other unicode characters. What is the relevant part of the code? Maybe I can poke around and see if I can get something to work.

[kylechui]

I imagine the relevant code should be in input.lua.

[Mango0x45]

Ah here is a hint. At the very start of M.get_add() in config.lua I threw in:

for k, _ in pairs(M.get_opts().surrounds) do
    print(k)
done

and I noticed that while all the other keys are being stored correctly (along with invalid_key_behavior and some defaults), the U+2019 quote mark that I entered is showing up not as it usually does as a right-single-quote, but as <e2><80><fe>X<99>. Looks like somewhere in the configuration unicode is not being properly handled.

[Mango0x45]

Looks like the issue occurs somewhere in the call to M.translate_opts() in M.merge_opts() in config.lua

UPDATE: The issue is the call to vim.api.nvim_replace_termcodes()

[kylechui]

Sorry for the delay; just writing to let you know that I really appreciate the PR and that it will be merged soon:tm:; I just want to figure out how to get your change autopublished to LuaRocks as well so it might take a little bit.

[Mango0x45]

No worries! I just hope this gets merged soon™ because I use multibyte characters all the time :)