Add support for StarCoder models

silvanmelchior / IncognitoPilot

An AI code interpreter for sensitive data, powered by GPT-4 or Code Llama / Llama 2.

MIT License

423 stars 44 forks source link

Add support for StarCoder models #18

Open osanseviero opened 1 year ago

osanseviero commented 1 year ago

Hi there! StarCoder from BigCode was trained for this kind of tasks, so having some documentation/support for it would be great.

Very nice project btw :fire:

ishaan-jaff commented 1 year ago

@osanseviero working on this PR for your issue: https://github.com/silvanmelchior/IncognitoPilot/pull/19

silvanmelchior commented 1 year ago

Thanks!

Regarding starcoder: Was this finetuned to work in chat-style interactions? When I had a first look at it, it appeared as if it's mostly for coding only / reading comments. Also from the size of the model I'm not sure if it can work well in this setup.

If someone can demo that it works well however, I'm very open to adding it!

osanseviero commented 1 year ago

There are multiple BigCode models (1.1B, 3B, 7B, and 15B), so there are smaller version indeed. As for which model, it depends on what exact behaviour one would want. https://twitter.com/lvwerra/status/1691127139314159628 is a good explanation

StarCoder+ is trained with more natural language
StarChat b is then instruct-tuned. -> see this quick demo https://huggingface.co/spaces/HuggingFaceH4/starchat-playground

silvanmelchior commented 1 year ago

I tried it out a bit on HF, looks nice!

Definitely sth to consider, or maybe just in general a better explanation on how to add more models.