search

Exafunction / codeium.el

Free, ultrafast Copilot alternative for Emacs
https://www.codeium.com
MIT License
412 stars 13 forks source link

Not seeing completions in some cases #13

Closed ksqsf closed 1 year ago

ksqsf commented 1 year ago

I tried to use codeium.el on a python-ts-mode buffer, but no completions show up, and *codeium-log* contains:

I0218 02:17:23.152879 28034 document.go:89] Unspecified language received enry language 
E0218 02:17:23.152896 28034 document.go:94] Unable to resolve enry language

When I turned on python-mode, it works fine. I suppose codeium.el could be amended:

https://github.com/Exafunction/codeium.el/blob/094b3cc6fca24b75d6571988e69f392336b2f317/codeium.el#L206-L212

(The default value of this list seems surprisingly limited!)

fortenforge commented 1 year ago

We definitely should expand this list, but you should still be able to get completions on languages that are not on this list. This list just helps us identify the specific language being used if Enry is unable to detect it correctly. Identifying the language lets us give more accurate completions.

Can you paste more of the output of *codeium-log* so we can figure out why you weren't seeing any completions?

ksqsf commented 1 year ago

After codeium-reset, use this document

from typing import List

def merge_sort(arr: |

(| denotes the cursor)

The logs:

Process codeium killed: 9
http://localhost:54163/exa.language_server_pb.LanguageServerService/GetCompletions 6.74 secs status: 200 ((state . "CODEIUM_STATE_SUCCESS") (message . "Generated 4 completions"))
http://localhost:54163/exa.language_server_pb.LanguageServerService/Heartbeat 5.00 secs status: 200
http://localhost:54163/exa.language_server_pb.LanguageServerService/CancelRequest 0.10 secs status: 500 "no session state for apiKey 1z2a9cz5-0zb4-z500-84z6-3286zzz3563a, sessionId "
I0218 03:45:12.356959 43248 main.go:346] Starting language server manager with pid 43248
I0218 03:45:12.357738 43248 main.go:82] Manager acquired its lock file /var/folders/t1/hyszwdtx4x1bgr2fd4cjv4qc0000gn/T/codeium_4mgNYv/locks/manager.lock
I0218 03:45:12.362428 43248 main.go:198] Waiting for random port file in /var/folders/t1/hyszwdtx4x1bgr2fd4cjv4qc0000gn/T/codeium_4mgNYv/child_random_port_1676663112357927000_5777333039819117464 to be created
I0218 03:45:12.415733 43249 main.go:340] Starting language server process with pid 43249
I0218 03:45:12.432889 43249 api_server_client.go:74] proxyplease.proxy> No proxy provided. Attempting to infer from system.
2023/02/18 03:45:12 [proxy.Provider.readDarwinNetworkSettingProxy]: Automatic proxy is not enabled.
2023/02/18 03:45:12 [proxy.Provider.parseProxyInfo]: ProxyBypass="192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,127.0.0.1,localhost,*.local,timestamp.apple.com,sequoia.apple.com,seed-sequoia.siri.apple.com,", targetUrl=https://www.google.com, bypass=false
I0218 03:45:12.459386 43249 api_server_client.go:74] proxyplease.proxy> Inferred proxy from system: http://127.0.0.1:7890
I0218 03:45:12.459415 43249 api_server_client.go:76] Successfully dialed proxy
I0218 03:45:12.459555 43249 server.go:190] Successfully created API server client
I0218 03:45:12.473032 43249 server.go:197] Successfully initialized tokenizer
I0218 03:45:12.473130 43249 server.go:204] Successfully created completion provider
I0218 03:45:12.473186 43249 unleash.go:58] proxyplease.proxy> No proxy provided. Attempting to infer from system.
I0218 03:45:12.473192 43249 server.go:217] Child process attempting to acquire lock file /var/folders/t1/hyszwdtx4x1bgr2fd4cjv4qc0000gn/T/codeium_4mgNYv/locks/child_lock_1676663112358173000_5457503710778178636
2023/02/18 03:45:12 [proxy.Provider.readDarwinNetworkSettingProxy]: Automatic proxy is not enabled.
I0218 03:45:12.473337 43249 server.go:227] Child process acquired lock file /var/folders/t1/hyszwdtx4x1bgr2fd4cjv4qc0000gn/T/codeium_4mgNYv/locks/child_lock_1676663112358173000_5457503710778178636
I0218 03:45:12.474462 43249 server.go:106] Language server will attempt to listen on host 127.0.0.1
I0218 03:45:12.474958 43249 server.go:129] Language server listening on random port at 54163
2023/02/18 03:45:12 [proxy.Provider.parseProxyInfo]: ProxyBypass="192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,127.0.0.1,localhost,*.local,timestamp.apple.com,sequoia.apple.com,seed-sequoia.siri.apple.com,", targetUrl=https://www.google.com, bypass=false
I0218 03:45:12.487430 43249 unleash.go:58] proxyplease.proxy> Inferred proxy from system: http://127.0.0.1:7890
I0218 03:45:12.487447 43249 unleash.go:61] Initializing Unleash with production environment
I0218 03:45:12.490689 43249 proxy.go:94] proxyplease.connect> Proxy successfully established. No authentication was required.
I0218 03:45:12.490690 43249 proxy.go:94] proxyplease.connect> Proxy successfully established. No authentication was required.
I0218 03:45:12.564497 43248 main.go:224] Language server manager found random port 54163
I0218 03:45:12.564537 43248 main.go:228] Language server manager attempting to connect to language server at 127.0.0.1:54163
I0218 03:45:12.580061 43248 main.go:116] Fixing language server port at 54163
I0218 03:45:12.580083 43248 main.go:274] Language server manager successfully connected to new language server at 127.0.0.1:54163
I0218 03:45:13.283797 43249 unleash.go:76] Successfully initialized unleash
I0218 03:45:17.561849 43249 document.go:89] Unspecified language received enry language 
E0218 03:45:17.561877 43249 document.go:94] Unable to resolve enry language 
E0218 03:45:17.561739 43249 interceptor.go:27] /exa.language_server_pb.LanguageServerService/CancelRequest: no session state for apiKey 1z2a9cz5-0zb4-z500-84z6-3286zzz3563a, sessionId 
I0218 03:45:17.639264 43249 proxy.go:94] proxyplease.connect> Proxy successfully established. No authentication was required.
http://localhost:54163/exa.language_server_pb.LanguageServerService/Heartbeat 0.02 secs status: 200
http://localhost:54163/exa.language_server_pb.LanguageServerService/Heartbeat 0.02 secs status: 200
ksqsf commented 1 year ago

Well, now I can't even get python-mode to work. There are some additional logs from python-mode:

I0218 03:54:44.665347 43582 main.go:224] Language server manager found random port 54564
I0218 03:54:44.665410 43582 main.go:228] Language server manager attempting to connect to language server at 127.0.0.1:54564
I0218 03:54:44.679561 43582 main.go:116] Fixing language server port at 54564
I0218 03:54:44.679582 43582 main.go:274] Language server manager successfully connected to new language server at 127.0.0.1:54564
I0218 03:54:45.309231 43583 unleash.go:76] Successfully initialized unleash
http://localhost:54564/exa.language_server_pb.LanguageServerService/CancelRequest 0.06 secs status: 200
E0218 03:54:48.991247 43583 prompt.go:342] Error getting sibling functions: UseTreeSitter: no function parent found at offset 39
http://localhost:54564/exa.language_server_pb.LanguageServerService/GetCompletions 0.53 secs status: 408 "Post \"https://server.codeium.com:443/exa.api_server_pb.ApiServerService/GetCompletions\": context canceled"
E0218 03:54:49.126624 43583 prompt.go:342] Error getting sibling functions: UseTreeSitter: no function parent found at offset 40
http://localhost:54564/exa.language_server_pb.LanguageServerService/GetCompletions 1.26 secs status: 408 "Post \"https://server.codeium.com:443/exa.api_server_pb.ApiServerService/GetCompletions\": context canceled"
E0218 03:54:49.595238 43583 prompt.go:342] Error getting sibling functions: UseTreeSitter: no function parent found at offset 42
http://localhost:54564/exa.language_server_pb.LanguageServerService/GetCompletions 1.55 secs status: 200 ((state . "CODEIUM_STATE_SUCCESS") (message . "Generated 2 completions"))
E0218 03:54:49.978578 43583 prompt.go:342] Error getting sibling functions: UseTreeSitter: no function parent found at offset 43
http://localhost:54564/exa.language_server_pb.LanguageServerService/GetCompletions 0.95 secs status: 200 ((state . "CODEIUM_STATE_SUCCESS") (message . "Generated 2 completions"))
http://localhost:54564/exa.language_server_pb.LanguageServerService/CancelRequest 0.02 secs status: 200
E0218 03:54:51.988956 43583 prompt.go:342] Error getting sibling functions: UseTreeSitter: no function parent found at offset 43
http://localhost:54564/exa.language_server_pb.LanguageServerService/Heartbeat 0.02 secs status: 200
http://localhost:54564/exa.language_server_pb.LanguageServerService/Heartbeat 0.03 secs status: 200
http://localhost:54564/exa.language_server_pb.LanguageServerService/Heartbeat 0.03 secs status: 200
fortenforge commented 1 year ago

OK interesting. So the GetCompletions requests are succeeding and the package is receiving completions, but you're not seeing them. Will look into it.

Alan-Chen99 commented 1 year ago

Do you by any chance use company-mode? company caches completion requests, which is empty when you interrupt codeium.el (by pressing any key)

ksqsf commented 1 year ago

No, I'm using corfu.

There are some more observations: when I first press C-M-i to request completions:

image

(sorry for screenshots)

the requests all somehow get stuck, and it can take arbitrarily long time here; but when I do anything, the responses are suddenly received, along with some other errors:

http://localhost:59921/exa.language_server_pb.LanguageServerService/Heartbeat 0.02 secs status: 200
http://localhost:59921/exa.language_server_pb.LanguageServerService/GetCompletions 117.23 secs status: 200 ((state . "CODEIUM_STATE_SUCCESS") (message . "Generated 3 completions"))
http://localhost:59921/exa.language_server_pb.LanguageServerService/GetCompletions 116.76 secs status: 200 ((state . "CODEIUM_STATE_SUCCESS") (message . "Generated 3 completions"))
http://localhost:59921/exa.language_server_pb.LanguageServerService/Heartbeat 116.13 secs status: 200
http://localhost:59921/exa.language_server_pb.LanguageServerService/CancelRequest 0.08 secs status: 200
E0218 06:32:24.448708 38264 prompt.go:342] Error getting sibling functions: UseTreeSitter: no function parent found at offset 43
E0218 06:32:24.896654 38264 prompt.go:342] Error getting sibling functions: UseTreeSitter: no function parent found at offset 43
Alan-Chen99 commented 1 year ago

Hmm codeium runs things like logging/ returning completions on a run-with-idle-timer. For some reason the timer does not fire, Can you think of why that would be the case?

Alan-Chen99 commented 1 year ago

alternatively, perhaps something causes your (input-pending-p) to be always t in those 2 minutes? (maybe the debugger?)

Alan-Chen99 commented 1 year ago

try adding this to your config and see if it helps?

(defun my-codeium-defer-until-no-input (state tracker func &optional args)
    (when (eq tracker (codeium-state-alive-tracker state))
        (with-local-quit
            (apply func args))))
(advice-add 'codeium-defer-until-no-input :override #'my-codeium-defer-until-no-input)
ksqsf commented 1 year ago

No luck with removing (input-pending-p). It returns correct values while I'm debugging.

I found that the invocation of codeium-request-callback is somehow blocked on user input: it does nothing until I generate some events (e.g. keyboard input or mouse clicks), and I can see the completions in the debug logs when the callback is invoked. I'm not sure if url-retrieve is the culprit here...

BTW, I'm using Emacs 29. Are you using 28 or older versions?

ksqsf commented 1 year ago

I cannot reproduce this problem with emacsq. So it's more likely an issue in my config.

Sorry for the disturbance. Feel free to close this issue.

If I find anything valuable to add, I will comment here.

Alan-Chen99 commented 1 year ago

the requests all somehow get stuck, and it can take arbitrarily long time here; but when I do anything, the responses are suddenly received, along with some other errors:

http://localhost:59921/exa.language_server_pb.LanguageServerService/Heartbeat 0.02 secs status: 200
http://localhost:59921/exa.language_server_pb.LanguageServerService/GetCompletions 117.23 secs status: 200 ((state . "CODEIUM_STATE_SUCCESS") (message . "Generated 3 completions"))
http://localhost:59921/exa.language_server_pb.LanguageServerService/GetCompletions 116.76 secs status: 200 ((state . "CODEIUM_STATE_SUCCESS") (message . "Generated 3 completions"))
http://localhost:59921/exa.language_server_pb.LanguageServerService/Heartbeat 116.13 secs status: 200
http://localhost:59921/exa.language_server_pb.LanguageServerService/CancelRequest 0.08 secs status: 200
E0218 06:32:24.448708 38264 prompt.go:342] Error getting sibling functions: UseTreeSitter: no function parent found at offset 43
E0218 06:32:24.896654 38264 prompt.go:342] Error getting sibling functions: UseTreeSitter: no function parent found at offset 43

Reread this; do the "Error getting sibling functions" also show up late? If that is the case, could it be that your process outputs are somehow buffered?

fortenforge commented 1 year ago

Closing due to inactivity. Feel free to reopen if you ever discover what the issue in your config was.

bcc32 commented 9 months ago

Several users at my site ran into an issue that had very similar symptoms to this one, in particular with the completion response not appearing to be received until the user presses a key.

We believe this may be caused by the following scenario:

  1. User's config enables various mode line, margin, etc. visual elements that take some time to redisplay. So (redisplay) reliably takes longer than 1ms.
  2. codeium-request-synchronously has a tight sit-for loop that waits for codeium-delay seconds (1 millisecond by default) for the response to be available.
  3. sit-for calls redisplay, but by the time redisplay finishes, 1ms has already elapsed, so Emacs never recognizes that network process output is available to be read. (I think this is a bug in sit-for---its documentation at least suggests that it should only start counting time after redisplay complets).
  4. Emacs sits idly, not reading the Codeium response, until the user causes some keyboard or mouse event (which interrupts the loop because input-pending-p now returns t). input-pending-p only checks for command input, not for subprocess events.

In our case, we were able to mitigate this by simply setting codeium-delay to a larger number, 0.1, that reliably exceeds the duration of redisplay. This shouldn't affect responsiveness of Codeium completion results, since sit-for will return once network input is available.

Perhaps it would be reasonable to increase the default value of codeium-delay somewhat. Common setups (like Spacemacs) can cause redisplay to take longer than a millisecond, which ought to be reasonable. (Separately we are considering reporting the bug/documentation issue in sit-for to upstream Emacs devs).

fortenforge commented 9 months ago

Thanks Aaron, we'll look into raising the value of codeium-delay