Support Starcoder models for FIM

twinnydotdev / twinny

The most no-nonsense, locally or API-hosted AI code completion plugin for Visual Studio Code - like GitHub Copilot but completely free and 100% private.

https://twinny.dev

MIT License

2.58k stars 143 forks source link

Support Starcoder models for FIM #141

Closed pacman100 closed 6 months ago

pacman100 commented 6 months ago

Is your feature request related to a problem? Please describe. Currently, for FIM, codellama, deepseekcoder and stable-code models are only supported. Starcoder has recently released version 2 of their models which are very performant. It would be great to support Starcoder models via Ollama. Ollama already has Starcoder 1 model on its model repository. Screenshot 2024-02-28 at 8 11 57 PM

rjmacarthy commented 6 months ago

Hey it seems like starcoder has the same fim tokens as stablecode therefore it should work already if you choose stablecode as your format. I will update the name to match soon, please let me know. If the new models have new tokens please point me towards the resource.

Thanks.

pacman100 commented 6 months ago

Hello @rjmacarthy,

When I specifically choose stable-code template, it is using the codellama FIM tokens instead of Stablecode/Starcoder tokens as seen below:

pacman100 commented 6 months ago

Raised PR https://github.com/rjmacarthy/twinny/pull/143 to fix this.

pacman100 commented 6 months ago

Hey it seems like starcoder has the same fim tokens as stablecode therefore it should work already if you choose stablecode as your format. I will update the name to match soon, please let me know. If the new models have new tokens please point me towards the resource.

Thank you for the pointer! Yes, I verified the same.

I quickly raised this PR https://github.com/rjmacarthy/twinny/pull/144 to add support for starcoder models.

rjmacarthy commented 6 months ago

Hi @pacman100 the new version should be live now. Thanks for your contribution.

rjmacarthy commented 6 months ago

This is completed, so closing.