feat: Support Stable Code Instruct 3B

carlrobertoh / CodeGPT

The leading open-source AI copilot for JetBrains. Connect to any model in any environment, and customize your coding experience in any way you like.

https://codegpt.ee

Apache License 2.0

1.01k stars 213 forks source link

feat: Support Stable Code Instruct 3B #552

Closed reneleonhardt closed 4 months ago

reneleonhardt commented 4 months ago

https://stability.ai/news/introducing-stable-code-instruct-3b https://huggingface.co/stabilityai/stable-code-instruct-3b https://huggingface.co/bartowski/stable-code-instruct-3b-GGUF https://ollama.com/library/stable-code

Prompt seems identical to Code Qwen, but I had to add <|im_end|> as stop token. Can you try it out, FIM isn't possible with the instruct model, right?

Too bad we can't run 2 models after each other like I proposed...

User enters chat window focus, start Instruct model automatically (stop FIM if running)
User leaves chat window focus, start Code Completion model automatically (stop Instruct if running)

For example https://huggingface.co/google/codegemma-7b https://huggingface.co/google/codegemma-7b-it https://huggingface.co/meta-llama/Meta-Llama-3-8B https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct

carlrobertoh commented 4 months ago

FIM seems to be possible with this model. However, you need to specify the correct FIM template in order for this to work.

It supports code completion, FIM tasks, and understands nuanced programming queries, enabling developers to work more efficiently and effectively with less effort.

https://stability.ai/news/introducing-stable-code-instruct-3b

reneleonhardt commented 4 months ago

I reused the CodeQwen infill template (like the prompt template), it seems to work.

In addition I sorted the LLaMA models in settings because it seems fairer than appending all new models to the end where no one can see them... it would be cool to persist the plugin version between updates/reinstall, then we could mark all new ones (according to each user personally). Similar to the changelog, now IntelliJ showed only What's New 2.7.1, but I was coming from 2.6.3, so I can't see what changed in 2.7.0... strange "changelog" only showing one change/release 🤷

carlrobertoh commented 4 months ago

it would be cool to persist the plugin version between updates/reinstall, then we could mark all new ones (according to each user personally).

Not sure I follow. What do you mean by persisting the plugin version between updates?

Similar to the changelog, now IntelliJ showed only What's New 2.7.1, but I was coming from 2.6.3, so I can't see what changed in 2.7.0... strange "changelog" only showing one change/release 🤷

Yeah, it probably just shows the latest/recent changes. I had to patch a fix really quickly due to some backward compatibility issues.