Closed liuzhenghua closed 6 months ago
This situation does occur sometimes, and we have also noticed this phenomenon where there sometimes is an extra space after <fim_middle>
(not <fim_prefix>
tokens). Currently, a relatively convenient solution is to try to avoid this situation through pre-processing and post-processing, especially to prevent indent at the first word during inference.
An extra space after
fim prefix ending with corrent indent or not with \n,thethe problem would not occured
I'm getting this problem using Continue in vscode when trying to use this model. The problem does not occur with e.g. codellama.
Description
In FIM mode, an extra space is added at the beginning of the first line if is ends with \n.
How to repeat
When executing the above code, ensure that ends with a blank line. If the indentation of the whitespace characters is incorrect, an extra space will always be added to the first line of the resulting code.