[VS Code] Connot reach the server

[596192804]

• refact-self-hosting 1.3.0
• refact-vscode:v2.0.118

The Self-hosted Refact and the model ran successfully. I also logged into refact-vscode. However, when I tried code completion, it failed with Cannot reach the server. POST http://127.0.0.1:9098/v1/code-completion timed out after 20000 ms.

[olegklimov]

Could be something simple, for example the model might be still loading.

We also have this issue: self-hosted config changes is slow to reach the plugins, https://github.com/smallcloudai/refact/issues/158

[olegklimov]

...it's still bad that you hit it, and we want to know what happened, maybe we can convert it to something we can fix.

[596192804]

I can be certain that the model is up and running because when I execute the following on the server:

curl -H 'Content-Type: application/json' -H 'Accept: application/json' http://127.0.0.1:8008/v1/completions -d '
{
    "model": "deepseek-coder/1.3b/base",
    "prompt": "#write a quick sort",
    "temperature": 0.9,
    "top_p": 1.0,
    "n": 1,
    "max_tokens": 128,
    "stream": false,
    "presence_penalty": 0.0,
    "frequency_penalty": 0.0
}'

It immediately returns:

{"id": "comp-7rzmNMJMmhcv", "object": "text_completion", "status": "completed", "created": 1704849954.9084785, "uploaded": 1704849957.306223, "generated_tokens_n": 127, "model": "deepseek-coder/1.3b/base", "choices": [{"index": 0, "logprobs": null, "finish_reason": "length", "text": " program\n\n\ndef quick_sort(arr,l,r):\n    if l>=r:\n        return\n    pivot=arr[r]\n    i=l\n    for j in range(l,r):\n        if arr[j]<pivot:\n            arr[i],arr[j]=arr[j],arr[i]\n            i+=1\n    arr[i],arr[r]=arr[r],arr[i]\n    quick_sort(arr,l,i-1)\n    quick_sort(arr,"}]

This indicates that the request has been successfully processed, and the generated response contains the completion for the provided prompt.

[596192804]

I think the issue might be related to the plugin. When I tried to locate the service occupying port 9098, I found a service named "refact-lsp." Could it be the problem with this service?

[olegklimov]

Well it definitely works, based on your test! You can take a look at the logs:

tail -f -n 1000 ~/.cache/refact/logs/rustbinary.2024-01-11

And it can still be https://github.com/smallcloudai/refact/issues/158 which we really need to fix.

[596192804]

Thank you for your prompt reply! I tried the commands you provided, but it failed because there were no logs in the ~/.cache/refact/ directory. The result of executing ll ~/.cache/refact/ is as follows:

drwxr-xr-x 3 root root 4096 Jan  9 06:07 ./
drwxr-xr-x 1 root root 4096 Jan  9 07:24 ../
drwxr-xr-x 4 root root 4096 Jan  9 06:07 telemetry/

[596192804]

I also ran find / -name *rustbinary*, but the result came up empty.

[596192804]

I apologize for my earlier mistake. After running the command locally, here are the results:

2024-01-11T13:20:54.251516Z INFO refact_lsp::caps:331: reading caps from http://10.12.3.31:8008/coding_assistant_caps.json
2024-01-11T13:20:54.251920Z INFO refact_lsp::caps:347: caps 1 completion models
2024-01-11T13:20:54.251930Z INFO refact_lsp::caps:348: caps default completion model: "deepseek-coder/1.3b/base"
2024-01-11T13:20:54.251934Z INFO refact_lsp::caps:349: caps 1 chat models
2024-01-11T13:20:54.251936Z INFO refact_lsp::caps:350: caps default chat model: "deepseek-coder/6.7b/instruct"
2024-01-11T13:20:54.251952Z INFO refact_lsp::global_context:88: background reload caps successful
2024-01-11T13:20:54.337078Z INFO refact_lsp::caps:331: reading caps from http://10.12.3.31:8008/coding_assistant_caps.json
2024-01-11T13:20:54.337286Z INFO refact_lsp::caps:347: caps 1 completion models
2024-01-11T13:20:54.337290Z INFO refact_lsp::caps:348: caps default completion model: "deepseek-coder/1.3b/base"
2024-01-11T13:20:54.337293Z INFO refact_lsp::caps:349: caps 1 chat models
2024-01-11T13:20:54.337294Z INFO refact_lsp::caps:350: caps default chat model: "deepseek-coder/6.7b/instruct"
2024-01-11T13:20:54.337317Z INFO refact_lsp::global_context:124: quick load caps successful
2024-01-11T13:20:54.337365Z INFO refact_lsp::http::utils:34: http://127.0.0.1:9099/caps completed in 108.6324ms
2024-01-11T13:21:53.976084Z INFO refact_lsp::telemetry::basic_transmit:103: basic telemetry sending starts
2024-01-11T13:21:56.646808Z INFO refact_lsp::http::utils:17: http://127.0.0.1:9099/caps starts
2024-01-11T13:21:56.646853Z INFO refact_lsp::http::utils:34: http://127.0.0.1:9099/caps completed in 28.5µs
2024-01-11T13:22:01.179206Z INFO refact_lsp::http::utils:17: http://127.0.0.1:9099/chat starts
2024-01-11T13:22:01.181634Z INFO refact_lsp::cached_tokenizers:31: downloading tokenizer "https://huggingface.co//models/deepseek-coder-6.7b-instruct/resolve/main/tokenizer.json" to C:\Users\59619\AppData\Local\Temp\4e8f8257-751a-4a6c-91f6-3ad3312b7da1...

[olegklimov]

Looks alright! Should work if it can download the tokenizer etc.

[596192804]

I seem to have found the reason. It's because the URL "https://huggingface.co//models/deepseek-coder-6.7b-instruct/resolve/main/tokenizer.json" doesn't exist.

[596192804]

That's strange. The correct URL should be https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct/resolve/main/tokenizer.json.

[596192804]

I've figured out the reason! It's because I previously used offline model downloads, and I modified the model_path in /usr/local/lib/python3.10/dist-packages/known_models_db/refact_known_models/huggingface.py.

[596192804]

Thank you very much for your help！！！

[olegklimov]

Thanks for trying stuff!