Code completion not working

[andremald]

Hi! I am trying to use the tool but somehow the code completion is not working. The chat functionality works just fine so I am quite sure I configured the connectors properly. Unfortunately, I couldn't find any logging. Hence I am not even sure whether the completion request is being sent.

It tried the following models: codegemma, codellama and starcoder. ( always the fim version)

The path is /api/generate

Though it is also not working in a host vs code, I normally work inside a dev container one. Hence, I updated the hostname to host.docker.internal

As I said, the chat functionality works just fine. I am wondering what could be the issue with the coding completion one?

[rjmacarthy]

Hello,

Please confirm all settings used for FIM completion providers. Also, please enable the debugging information in the extension settings and tick enable logging, then go to Help -> Toggle Developer Tools inside Visual Studio Code to look out for any errors.

Many thanks,

[andremald]

Hi! While debugging I could see that there is a request being sent if I use the chat functionality. However, nothing shows up in the console for the code completion , even when I request it with Option + \ ( I am a mac user ).

I also found a "Problem creating default templates "/root/.twinny/templates""

Can it be that's the issue? No template-> no FIM?

[rjmacarthy]

Hey, that shouldn't be an issue for fim as they are built in. Please provide all the provider configuration settings as previously requested.

[cold-eye]

[andremald]

Type: FIM Fim template: codegemma Provider: ollama Protocol: http Model name: codegemma:2b Hostname: host.docker.internal Port: 11434 Path: /api/generate

As I mentioned in the previous message, I don't get request in the console as you do ( based on your photo )

EDIT: out of curiosity I did a ls at /root/.twinny/templates and cat in the fim *.hbs files ( there were two: fim.hbs and fim-system.hbs )

fim-system.hbs is empty.

fim.hbs contains the following

<PRE>{{{prefix}}} <SUF>>{{{sufix}}} <MID>

Hope that rings a bell. I would expect to have either more templates in a file or more template files.

EDIT 2: After staying stuck in the train I had the chance to 1) check your repo with more care, 2) debug a bit further. With regards to 1: Just ignore the message about the *.hbs files. I already understand that what you meant by "built-in".

With regards to 2: despite the fact that I don't get any logs about the request being sent, like I get when using the chat functionality, I do get the following.

2024-05-07 23:36:04.369 [info] [KeybindingService]: / Soft dispatching keyboard event
2024-05-07 23:36:04.369 [info] [KeybindingService]: \ Keyboard event cannot be dispatched
2024-05-07 23:36:04.369 [info] [KeybindingService]: / Received  keydown event - modifiers: [alt], code: AltRight, keyCode: 18, key: Alt
2024-05-07 23:36:04.370 [info] [KeybindingService]: | Converted keydown event - modifiers: [alt], code: AltRight, keyCode: 6 ('Alt')
2024-05-07 23:36:04.370 [info] [KeybindingService]: \ Keyboard event cannot be dispatched in keydown phase.
2024-05-07 23:36:04.408 [info] [KeybindingService]: / Soft dispatching keyboard event
2024-05-07 23:36:04.408 [info] [KeybindingService]: | Resolving alt+[Backslash]
2024-05-07 23:36:04.408 [info] [KeybindingService]: \ From 1 keybinding entries, matched editor.action.inlineSuggest.trigger, when: editorTextFocus && !editorReadonly, source: user extension rjmacarthy.twinny.
2024-05-07 23:36:04.408 [info] [KeybindingService]: / Received  keydown event - modifiers: [alt], code: Backslash, keyCode: 220, key: «
2024-05-07 23:36:04.408 [info] [KeybindingService]: | Converted keydown event - modifiers: [alt], code: Backslash, keyCode: 93 ('\')
2024-05-07 23:36:04.408 [info] [KeybindingService]: | Resolving alt+[Backslash]
2024-05-07 23:36:04.409 [info] [KeybindingService]: \ From 1 keybinding entries, matched editor.action.inlineSuggest.trigger, when: editorTextFocus && !editorReadonly, source: user extension rjmacarthy.twinny.
2024-05-07 23:36:04.409 [info] [KeybindingService]: + Invoking command editor.action.inlineSuggest.trigger.
2024-05-07 23:36:04.586 [info] [KeybindingService]: + Ignoring single modifier alt due to it being pressed together with other keys.

Attention to 2024-05-07 23:36:04.409 [info] [KeybindingService]: + Invoking command editor.action.inlineSuggest.trigger.

Hope it rings a bell now. I went through your code and though I am not a typescript programmer, I could follow most of it and it looks alright. I am somewhat clueless.

[rjmacarthy]

I would recommend trying codellama:7b-code to see if it works.

[andremald]

Just gave a try, still nothing:

Settings:

Edit: obviously with hostname replaced by localhost

Console after a successful call to the chat api and several "Option + \ " in a python file:

[dishbrains]

I have the same problem as you andremald, i.e. chat is working fine but FIM does nothing

anyway, and input on how to fix this would be appreciated. this otherwise great ext is not usable for me like that

[oregonpillow]

Same problem here. All settings correct. 13b or 7b, doesn't matter. Only chat seems to work. I see the robot icon loading when i start coding, but no autocomplete prompts ever how

[localbarrage]

Failing for me too. I can see the message being received by the provider, but no response and no error. I am using Aphrodite's openai api server. I have tried different providers, yet none give a resopnse.

[localbarrage]

My issue might partly related to there not being an actual supported OpenAi provider. I setup a litellm proxy to forward to my model and I am still not getting any completions.

[hitzhangjie]

+1

[jleivo]

Hi.

I have the same issue: chat works, FIM doesn't, no matter what I do in the configurations. Setup: Ollama on a separate server, coding done within WSL => twinny is in WSL

I was looking at the developer tools. as suggested, and when I was writing in the VScode I saw this in the Developer console

ERR memory access out of bounds: RuntimeError: memory access out of bounds at wasm://wasm/000bc226:wasm-function[254]:0x2b979 at Parser.parse (/home/juleivo/.vscode-server/extensions/rjmacarthy.twinny-3.11.39/out/index.js:2:218649) at t.CompletionProvider.provideInlineCompletionItems (/home/juleivo/.vscode-server/extensions/rjmacarthy.twinny-3.11.39/out/index.js:2:123675) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async B.provideInlineCompletions (/home/juleivo/.vscode-server/bin/dc96b837cf6bb4af9cd736aa3af08cf8279f7685/out/vs/workbench/api/node/extensionHostProcess.js:155:108949)

After receiving this error I went to the WSL to ~/.vscode and deleted all twinny related folders. Started VS code and installed fresh copy of twinny. Now it works. I had twinny 3.11.10 and 3.11.31 on the host, now I have 3.11.39 and all is good again. I'll repeat this on my work computer later on, to see if this for some reason fixes the issue...

[NeoMatrixJR]

Same issues here. Ollama is running in docker on an external server. Chat works, no FIM. Have tried other extensions (continue dev at least) and get FIM...not good...but it at least does something....so I know it's not an issue with Ollama. EDIT: I dropped this back to a 3.10.* version, kept everything as stock as possible...installed the default codellama models and set it to the IP of my server....now it seems to work. I'll try and tweak it later to see if it's a plugin version issue, a model issue....???

[KizzyCode]

Same issue, OS is macOS, provider is ollama with starcoder:3b – ollama gets request and does computation, but whatever is computed does not show up in VSCode...

Weird is that some time ago, the extension worked flawlessly and I did no manual change except installing auto-updates.

[yuhanj2]

I found that setting Folding Range Provider in vscode settings to twinny would solve this issue for me, not sure if it will work for everyone.

[KizzyCode]

@yuhanj2 Thx, that workaround works for me too!

[GreatApo]

Same issue here. Could it be bugging out due to other extensions?

[rjmacarthy]

Hello all, do we have any indication of what is broken? I have been using the extension myself recently and it's working with latest version of code. Many thanks.

❯ code -v
1.90.2
5437499feb04f7a586f677b155b039bc2b3669eb
x64

[swedenSnow]

First of all, thank you for all your work @rjmacarthy 🙌🏽

I'm having the same issue, have a setup with ollama and just updated VSC to 1.90.2 and ollama 0.1.48 llama3 for chat ( working ) dolphincoder for FIM (I've tried with codellama, same issue)

[rjmacarthy]

Hello, I think starcoder 2 is not good for fim completion I tried it. Please try one of the recommended models in the docs. Many thanks.

[swedenSnow]

Thanks for the heads up. Was working ok but now looking at the logs it seems trying to get a model I dont have locally anymore: "model": "deepseek-coder:1.3b-base-fp16", even when I changed provider to codellama.

[rjmacarthy]

Maybe try to restart ide.

[swedenSnow]

I have. Its a bit confusing now: I installed the deepseek model as mentioned above, and then my FIM started working althogh still not requiring the model I have entered as provider (codellama) but deepseek again 🤔 Twinny Stream Debug Streaming response from 0.0.0.0:11434. Request body: "model": "deepseek-coder:1.3b-base-fp16", "prompt": "<｜fim▁begin｜>/*/\n\n/ Language: Typescript React (typescriptreact) /\n/ File uri

EDIT: Resetting providers or deleting and adding again seems to work. Funny how I must been using deepseek all this time until I deleted it, and thought I was using starcoder2 :) Its really not good for FIM as you pointed out

[GreatApo]

Hello all, do we have any indication of what is broken? I have been using the extension myself recently and it's working with latest version of code. Many thanks.

To start with, thank you for you work!

I was getting the following error upon writing code:

I disabled-enabled the extension, and now it prints the request, but nothing happens:

Then, after a while, the following errors appear:

The chat works...

VSCode info:

Edit: Upon resetting the providers, something looks like it going on, but file errors start popping up again: (mind that my code & ollama are on a local network server) Edit 2: Running ollama locally (same machine) seems to work okish.

[fredrikburmester]

I also have the issue of FIM not working. Running Ollama locally on M3 Mac. VSCode Version: 1.91.0

[fredrikburmester]

This is an error in the console:

ERR Cannot read properties of undefined (reading 'apply'): TypeError: Cannot read properties of undefined (reading 'apply')
    at wasmImports.<computed>.stub.e.<computed> (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:177464)
    at null.<anonymous> (wasm://wasm/0003dbda:1:39736)
    at null.<anonymous> (wasm://wasm/0003dbda:1:35226)
    at postInstantiation (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:178474)
    at /Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:178735
    at t.getParser (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:114775)
    at t.CompletionProvider.provideInlineCompletionItems (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:123581)
    at J.provideInlineCompletions (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:154:119139)

Also:

ERR Parsing failed: Error: Parsing failed
    at Parser.parse (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:218987)
    at t.CompletionProvider.provideInlineCompletionItems (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:123675)
    at J.provideInlineCompletions (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:154:119139)

[arungithub9]

This is an error in the console:

ERR Cannot read properties of undefined (reading 'apply'): TypeError: Cannot read properties of undefined (reading 'apply')
    at wasmImports.<computed>.stub.e.<computed> (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:177464)
    at null.<anonymous> (wasm://wasm/0003dbda:1:39736)
    at null.<anonymous> (wasm://wasm/0003dbda:1:35226)
    at postInstantiation (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:178474)
    at /Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:178735
    at t.getParser (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:114775)
    at t.CompletionProvider.provideInlineCompletionItems (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:123581)
    at J.provideInlineCompletions (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:154:119139)

Also:

ERR Parsing failed: Error: Parsing failed
    at Parser.parse (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:218987)
    at t.CompletionProvider.provideInlineCompletionItems (/Users/username/.vscode/extensions/rjmacarthy.twinny-3.11.43/out/index.js:2:123675)
    at J.provideInlineCompletions (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:154:119139)

I am facing the same error with VS Code Ver 1.90.2.

[rjmacarthy]

I just released a new version which adds a try catch to the parsing of the document to attempt to fix this problem. Please let me know if it helps, if not it would be helpful to know what file/language/extension is being used when this error appears. Many thanks.

[hrush98]

Hey!

Having some trouble with autocomplete. I activated the developer tools and can see the completion requests being made, but it appears that the suggestions aren't being shown in my editor; that is I'm not seeing the inline completions. Chat works fine, wondering if you or anyone has encountered issues with it showing in the editor?

Using the stock codellama-7b settings.

It works sporadically, which is also confusing. Tried all suggestions posted here already. Great tool when it works for me though

Thanks!

[Extension Host] [twinny] 
***Twinny Stream Debug***
Streaming response from localhost:11434.
Request body:
{
  "model": "codellama:7b-code",
  "prompt": "<PRE>'''''' \n\n''' Language: Python (python) '''\n''' File uri: file:///home/local_methods/data_viewer.py (python) '''\nimport json\nimport pandas as pd\nimport igraph as ig\n\n\n# Alternatively, load the JSON file in chunks if it's very large\ndf = pd.read_json('./Data/dataset_8-13-24.json', lines=True, chunksize=10)\n# This will load the first 1000 lines as a DataFrame\n\nwith open('./Data/dataset_8-13-24.json') as f: \n    data = json.load(f)\n    \nfirst_1000_entries = data[:1000]\n\nwith open('first_1000_entries.json', 'w') as output_file:\n    json.dump(first_1000_entries, output_file, indent=4)\n\ndf = pd.DataFrame(data) \n# Check the structure of the DataFrame\n#print(df.head())  # View the first few rows of the DataFrame\n\nfirst = df.head(n = 20) \nfirst.to_csv('FirstFew.csv', index = False)\n\n\ndf =  <SUF> \n\n\n\n\n#print(df.columns) <MID>",
  "stream": true,
  "keep_alive": "5m",
  "options": {
    "temperature": 0.2,
    "num_predict": 512
  }
}

Request options:
{
  "hostname": "localhost",
  "port": 11434,
  "path": "/api/generate",
  "protocol": "http",
  "method": "POST",
  "headers": {
    "Content-Type": "application/json",
    "Authorization": ""
  }
}

Number characters in all messages = undefined

console.ts:137 [Extension Host] [twinny] *** Twinny completion triggered for file: file:///home/hmrush/Desktop/Lightning%20Wrok/amboss-rl/local_methods/data_viewer.py ***
      Original completion: pd.read_json('./Data/dataset_8-13-24.json')
      Formatted completion: pd.read_json('./Data/dataset_8-13-24.json')
      Max Lines: 30
      Use file context: true
      Completed lines count 1
      Using custom FIM template fim.bhs?: false

Chat works fine:

[Extension Host] [twinny] 
***Twinny Stream Debug***
Streaming response from 0.0.0.0:11434.
Request body:
{
  "model": "codellama:7b-instruct",
  "stream": true,
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful, respectful and honest coding assistant.\nAlways reply using markdown.\nBe clear and concise, prioritizing brevity in your responses.\nFor code refactoring, use markdown with appropriate code formatting."
    },
    {
      "role": "user",
      "content": "chat works fine"
    }
  ],
  "keep_alive": "5m",
  "options": {
    "temperature": 0.2,
    "num_predict": 512
  }
}

Request options:
{
  "hostname": "0.0.0.0",
  "port": 11434,
  "path": "/v1/chat/completions",
  "protocol": "http",
  "method": "POST",
  "headers": {
    "Content-Type": "application/json",
    "Authorization": "Bearer undefined"
  }
}

Number characters in all messages = 233`

[Hezkore]

I couldn't get a manual suggestion to trigger, not by the command nor the keybind. However, disabling auto suggestion made the manual trigger work just fine. "twinny.autoSuggestEnabled": false in settings.