vim.fn.json_encode fails on <89>

[jemag]

At first I thought it was related to https://github.com/jcdickinson/codeium.nvim/issues/20, but it doesn't seem like it.

Basically, in my command history I have the following (taken from main.shada file):

  @ Description_  Value
  - history type  CMD
  - contents      "set lines=100<89>"

When I am in the command window q:, editing something that will trigger completion will make codeium fail with the following:

Error detected while processing TextChangedI Autocommands for "*":
Error executing lua callback: Vim:E474: String "<89>
stack traceback:
    [C]: in function 'json_encode'
    ...g/.local/share/nvim/lazy/codeium.nvim/lua/codeium/io.lua:409: in function 'post'
    .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:146: in function 'request'
    .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:287: in function 'request_completion'
    ...ocal/share/nvim/lazy/codeium.nvim/lua/codeium/source.lua:103: in function 'complete'
    ...jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/source.lua:325: in function 'complete'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:289: in function 'complete'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:169: in function 'callback'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:219: in function 'autoindent'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:161: in function 'on_change'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/init.lua:313: in function 'callback'
    ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:49: in function 'emit'
    ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:23: in function <...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:22>

What seems to happen is that the param.body contains my history, which in turn contains this <89> char which will then fail the vim.fn.json_encode from : https://github.com/jcdickinson/codeium.nvim/blob/55fa67bd316e2a4d312b11d68a2c34f898925a7f/lua/codeium/io.lua#L403-L408

Not sure what is the best approach, but I am guessing that params.body probably needs to be sanitized before being passed to vim.fn.json_encode to prevent this kind of errors.

[jemag]

Seems like there is a non UTF-8 character in my history hence why it errors out.

Even if there is no sanitizing, it would probably be best to handle the error.

For now, I can probably work around with a fork and something like:

    local ok
    ok, params.body = pcall(vim.fn.json_encode, params.body)
    if not ok then
      return
    end

[jemag]

is there any issue preventing using a proper pcall with vim.fn.json_encode?

I would much rather have this change in upstream instead of me having to maintain a fork with the following changes to the post function:

        local ok, encoded_body = pcall(vim.fn.json_encode, params.body)
        if not ok then
            log.error('Could not json_encode body')
            f:close()
            return
        end
        f:write(encoded_body)

I would submit a PR but I think there's probably a cleaner way to go about this