Open gergap opened 1 month ago
Hi, I'm running deepseek-coder-v2:16b in ollama. I'm trying to make FIM working to get a code completion working in Vim using tab-completion (works e.g. with codellama). However, it outputs much more text then just the completion. Seems to ignore the FIM token.
Example prompt:
<|fim_begin|>int main(<|fim_hole|><|fim_end|>
Answer:
Here's the corrected and completed version of your code snippet for a simple C++ program that outputs "Hello, World!" to the console: ```cpp #include <iostream> using namespace std; int main() { cout << "Hello, World!" << endl; return 0; }
Expected Answer:
int argc, char *argv)
What I'm doing wrong? According to the docs the 16b model is trained with FIM.
hi, do you use model deepseek-coder-v2-lite-base? I use this model with hf to gernerate FIM is good.
and I think your fim template need space before words, like this:
<|fim▁begin|> <|fim▁hole|> <|fim▁end|>
You are right. The FIM tokens were wrong. Also the underscore is not a normal underscore. This works now.
let g:ollama_model = 'deepseek-coder-v2:16b-lite-base-q4_0'
let g:ollama_fim_prefix = '<|fim▁begin|>'
let g:ollama_fim_suffix = '<|fim▁hole|>'
let g:ollama_fim_middle = '<|fim▁end|>'
This is what the hexdump of the working token looks like:
$> echo '<|fim▁begin|>' | hexdump -C
00000000 3c ef bd 9c 66 69 6d e2 96 81 62 65 67 69 6e ef |<...fim...begin.|
00000010 bd 9c 3e 0a |..>.|
Hi, I'm running deepseek-coder-v2:16b in ollama. I'm trying to make FIM working to get a code completion working in Vim using tab-completion (works e.g. with codellama). However, it outputs much more text then just the completion. Seems to ignore the FIM token.
Example prompt:
Answer:
Expected Answer:
What I'm doing wrong? According to the docs the 16b model is trained with FIM.