search

Exafunction / codeium.nvim

A native neovim extension for Codeium
MIT License
658 stars 50 forks source link

Codeium throws errors on large files #20

Closed wonderbeel closed 2 months ago

wonderbeel commented 1 year ago

Hi, I am playing around with this plugin and found an issue: I have treesitter configured to disable text highlight in big files (in those cases it is too slow and makes the editor hang for a couple of seconds after each key stroke, falling back to the regex parser solves this issue) and it seems that doing this makes your plugin break with the following error

Error detected while processing TextChangedI Autocommands for "*":
Error executing lua callback: .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:405: Failed to spawn process: {
  _additional_on_exit_callbacks = {},
  _shutdown_check = <userdata 1>,
  _stderr_results = {},
  _stdout_results = {},
  _user_on_exit = <function 1>,
  args = { "-sSL", "-D", "/tmp/plenary_curl_2fe047dc.headers", "--compressed", "-X", "POST", "-H", "Content-Type: application/json", "--data-raw", "{\"editor_options\": {\"insert_spaces\": true, \"tab_size\": 2}, \"metadata\": {\"extension_name\": \"vim\", \"ide_name\": \"neovim\", \"request_id\": 3, \"api_key\": \"API_KEY_HERE\", \"extension_version\": \"1.1.32\", \"ide_version\": \"0.8.3\"}, \"document\": {\"editor_language\": \"sql\", \"language\": 41, \"line_ending\": \"\\n\", \"cursor_offset\": 1043599, \"text\": \"ENTIRE_FILE_HERE\"}

(api key and text have been cropped out just to be safe)

If I enable treesitter highlight again the error disappear but then the editor became unusable, it would be great to fix this issue.

Let me know if you need something else to help debug this issue :).

jcdickinson commented 1 year ago

I'm having trouble reproing this issue. How are you disabling TS? TSBufDisable?

wonderbeel commented 1 year ago

I am disabling it directly in the configuration like this (basically what treesitter suggests in their docs https://github.com/nvim-treesitter/nvim-treesitter/blob/master/doc/nvim-treesitter.txt#L117 )

      highlight = {
        enable = true,
        disable = function(_, buf)
          local max_filesize = 100 * 1024 -- 100 KB
          local ok, stats = pcall(vim.loop.fs_stat, vim.api.nvim_buf_get_name(buf))
          if ok and stats and stats.size > max_filesize then
            return true
          end
        end,
      }

Now that I reread the issue again it was slightly misleading, what I am actually doing is disabling the highlight module and fallback to the regex, not disabling the entire treesitter, sorry my bad.

jemag commented 1 year ago

Not sure if this is related, but also getting an error from the command window (q:), when editing text and completion is about to appear

Error detected while processing TextChangedI Autocommands for "*":
Error executing lua callback: Vim:E474: String "<89>
stack traceback:
    [C]: in function 'json_encode'
    ...g/.local/share/nvim/lazy/codeium.nvim/lua/codeium/io.lua:407: in function 'post'
    .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:146: in function 'request'
    .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:287: in function 'request_completion'
    ...ocal/share/nvim/lazy/codeium.nvim/lua/codeium/source.lua:103: in function 'complete'
    ...jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/source.lua:325: in function 'complete'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:289: in function 'complete'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:169: in function 'callback'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:219: in function 'autoindent'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:161: in function 'on_change'
    ...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/init.lua:313: in function 'callback'
    ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:49: in function 'emit'
    ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:23: in function <...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:22>

Edit: Not related to this, created separate issue

brunolpsousa commented 1 year ago

I got the same error, but in my case the problem was not in the treesitter highlight, but in the big files themselves.

I was having this problem on a file with around 3500 lines and when I deleted around 1000 lines the problem was gone. On the other hand, I took a plain text file and copy/pasted until I got to about 5000 lines and the problem started.

I was also able to reproduce this in .zshrc and Lua files.

Edit: the problem is E2BIG: argument list too long

Full error ``` Error detected while processing TextChangedI Autocommands for "*": Error executing lua callback: .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:405: Failed to spawn process: { _additional_on_exit_callbacks = {}, _shutdown_check = , _stderr_results = {}, _stdout_results = {}, _user_on_exit = , args = { "-sSL", "-D", "/run/user/1000/plenary_curl_3ffe4d44.headers", "--compressed", "-X", "POST", "-H", "Content-Type: application/json", "--data-raw", "{\"editor_options\": {\"insert_spaces\": true, \"tab_size\": 2}, \"metadata\": {\"request_id\": 4, \"ide_name\": \"neovim\", \"extension_name\": \"vim\", \"api_key\": \"my_api_key\", \"extension_version\": \"1.2.85\", \"ide_version\": \"0.9.2\"}, \"document\": {\"editor_language\": \"zsh\", \"language\": 0, \"line_ending\": \"\\n\", \"cursor_offset\": 1288, \"text\": \"a lot of text... command = "curl", enable_handlers = true, enable_recording = true, interactive = true, pid = "E2BIG: argument list too long", stderr = , stdin = , stdout = , user_data = {}, = <1>{ __index = , _create_uv_options = , _execute = , _pipes_are_closed = , _prepare_pipes = , _reset = , _shutdown = , _stop = , add_on_exit_callback = , after = , after_failure = , after_success = , and_then = , and_then_on_failure = , and_then_on_failure_wrap = , and_then_on_success = , and_then_on_success_wrap = , and_then_wrap = , chain = , chain_status = , co_wait = , is_job = , join = , new = , pid = , result = , send = , shutdown = , start = , stderr_result = , sync = , wait = } } stack traceback: .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:405: in function '_execute' .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:448: in function 'start' ....local/share/nvim/lazy/plenary.nvim/lua/plenary/curl.lua:288: in function 'post' ...o/.local/share/nvim/lazy/codeium.nvim/lua/codeium/io.lua:452: in function 'post' .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:146: in function 'request' .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:285: in function 'request_completion' ...ocal/share/nvim/lazy/codeium.nvim/lua/codeium/source.lua:142: in function 'complete' ...bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/source.lua:334: in function 'complete' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:299: in function 'complete' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:169: in function 'callback' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:229: in function 'autoindent' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:161: in function 'on_change' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/init.lua:338: in function 'callback' ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:49: in function 'emit' ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:23: in function <...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:22> stack traceback: [C]: in function 'error' .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:405: in function '_execute' .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:448: in function 'start' ....local/share/nvim/lazy/plenary.nvim/lua/plenary/curl.lua:288: in function 'post' ...o/.local/share/nvim/lazy/codeium.nvim/lua/codeium/io.lua:452: in function 'post' .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:146: in function 'request' .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:285: in function 'request_completion' ...ocal/share/nvim/lazy/codeium.nvim/lua/codeium/source.lua:142: in function 'complete' ...bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/source.lua:334: in function 'complete' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:299: in function 'complete' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:169: in function 'callback' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:229: in function 'autoindent' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:161: in function 'on_change' ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/init.lua:338: in function 'callback' ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:49: in function 'emit' ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:23: in function <...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:22> ```
waynebowie99 commented 9 months ago

I'm getting this same error.

I think my issue is related to @brunolpsousa where I'm getting this on large C# files.

Files that don't work:

  • 34KB with ~ 700 lines
  • 31KB with ~ 600 lines
  • 28KB with ~ 680 lines

Working files

  • 29KB with ~ 550 lines

It looks like the issue is tied more to the number of lines than anything in my case. Once the file gets to be around 600 lines it starts to fail in C#.

I'm running on Windows. I can provide any logs as needed

pqn commented 9 months ago

The fix is we probably have to switch the curl to accept data from stdin instead of passing it as an argument.

amoschristian commented 2 months ago

Still happening in Apr 2024 :(

mitya-murakhouski commented 2 months ago

Still happening in Apr 2024 :(

Same thing for me. Getting this error at big yml files, than any completion call on any file raise this error

Anthony-Fiddes commented 2 months ago

Edit: Realizing I still experience the issue if I open the too big file along with a smaller one. The codeium source appears to send other buffers as additional context. That's a big enough issue to where I guess I just have to disable the extension for now.

I was experiencing this issue too. I ended up configuring nvim-cmp per buffer, and excluding codeium on files over 100KB. From above comments, that number should probably be lower, but this works for me at the moment. You can see the change in my config here.

I referred to the nvim-cmp documentation on this page: https://github.com/hrsh7th/nvim-cmp/wiki/Advanced-techniques#disabling-completion-in-certain-contexts-such-as-comments

This obviously doesn't fix the root issue, and it would be nice if codeium worked in large files, but it will keep your neovim usable.

brunolpsousa commented 2 months ago

I made a PR to fix this. Can anyone test it and provide feedback?

Anthony-Fiddes commented 2 months ago

Yup, I can confirm that it works. I just changed Exafunction to brunolpsousa in my config and reinstalled the plugin. Awesome stuff, hope it gets merged soon!

Edit: Let me know if you're looking for any specific feedback. I tried it out with a file over 1MB.

CNich commented 2 months ago

I'm getting this problem too. It doesn't seem to be for big files necessarily, but seems to come when I have many tabs open/leave neovim on for long periods of time. I switched to brunolpsousa and ran Lazy sync but still get this problem, am I doing something wrong?

brunolpsousa commented 2 months ago

I'm getting this problem too. It doesn't seem to be for big files necessarily, but seems to come when I have many tabs open/leave neovim on for long periods of time. I switched to brunolpsousa and ran Lazy sync but still get this problem, am I doing something wrong?

You don't need to use my branch because it has already been merged. Make sure your Lazy config is not locked to a specific commit or any other version control. If it still doesn't work, I think it would be better to open a new issue and provide logs.

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.