FIM not working

[gergap]

Hi, I'm running deepseek-coder-v2:16b in ollama. I'm trying to make FIM working to get a code completion working in Vim using tab-completion (works e.g. with codellama). However, it outputs much more text then just the completion. Seems to ignore the FIM token.

Example prompt:

<|fim_begin|>int main(<|fim_hole|><|fim_end|>

Answer:

Here's the corrected and completed version of your code snippet for a simple 
C++ program that outputs "Hello, World!" to the console:

```cpp
#include <iostream>
using namespace std;

int main() {
    cout << "Hello, World!" << endl;
    return 0;
}
```

Expected Answer:

int argc, char *argv)

What I'm doing wrong? According to the docs the 16b model is trained with FIM.

[traveler-vee]

Hi, I'm running deepseek-coder-v2:16b in ollama. I'm trying to make FIM working to get a code completion working in Vim using tab-completion (works e.g. with codellama). However, it outputs much more text then just the completion. Seems to ignore the FIM token.

Example prompt:

<|fim_begin|>int main(<|fim_hole|><|fim_end|>

Answer:

Here's the corrected and completed version of your code snippet for a simple 
C++ program that outputs "Hello, World!" to the console:

```cpp
#include <iostream>
using namespace std;

int main() {
    cout << "Hello, World!" << endl;
    return 0;
}

Expected Answer:

int argc, char *argv)

What I'm doing wrong? According to the docs the 16b model is trained with FIM.

hi, do you use model deepseek-coder-v2-lite-base? I use this model with hf to gernerate FIM is good.

and I think your fim template need space before words, like this:

<｜fim▁begin｜> <｜fim▁hole｜> <｜fim▁end｜>

[gergap]

You are right. The FIM tokens were wrong. Also the underscore is not a normal underscore. This works now.

let g:ollama_model = 'deepseek-coder-v2:16b-lite-base-q4_0'
let g:ollama_fim_prefix = '<｜fim▁begin｜>'
let g:ollama_fim_suffix = '<｜fim▁hole｜>'
let g:ollama_fim_middle = '<｜fim▁end｜>'

[gergap]

This is what the hexdump of the working token looks like:

$> echo '<｜fim▁begin｜>' | hexdump -C
00000000  3c ef bd 9c 66 69 6d e2  96 81 62 65 67 69 6e ef  |<...fim...begin.|
00000010  bd 9c 3e 0a                                       |..>.|