⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains
This PR aims to improve granite3 models tab completion support. Granite3 is not FIM trained (unlike previous versions), so needs to use the holeFillerTemplate in AutocompleteTemplate.getTemplateForModel, instead of the default stableCodeFimTemplate.
Additionally, we found that smaller variants of granite3 models tend to sometimes return results that do not always satisfy the hole filler prompt instructions, typically repeating existing code in the response. Granite specific post-processing was added to postprocessCompletion to try to mitigate those misbehaviors.
Finally, I chose to remove extra lines prepended to this.writeLog(Completion:` statements, as I was having difficulty visually parsing the actual model output. Now we can really tell if extra lines were returned by the model.
Checklist
[x] No doc updates needed
Screenshots
[x] No visual changes
Testing
install granite3-dense:2b on ollama: ollama pull granite3-dense:2b
Description
This PR aims to improve granite3 models tab completion support. Granite3 is not FIM trained (unlike previous versions), so needs to use the holeFillerTemplate in AutocompleteTemplate.getTemplateForModel, instead of the default stableCodeFimTemplate. Additionally, we found that smaller variants of granite3 models tend to sometimes return results that do not always satisfy the hole filler prompt instructions, typically repeating existing code in the response. Granite specific post-processing was added to postprocessCompletion to try to mitigate those misbehaviors.
Finally, I chose to remove extra lines prepended to
this.writeLog(
Completion:` statements, as I was having difficulty visually parsing the actual model output. Now we can really tell if extra lines were returned by the model.Checklist
Screenshots
Testing
ollama pull granite3-dense:2b
the model returns:
The end of the prefix is then removed from the completion, so it can be inserted properly