New Goinfer provider

[synw]

I started to add support for a Goinfer provider here: https://github.com/synw/wingman

https://github.com/nvms/wingman/compare/main...synw:wingman:main

[nvms]

Nice! Really excited to try this.

[synw]

I added experimental support for templating and context window ( #13 , #17 ) for now in settings. Please check it out and tell me what you think. Here are the steps to try it out:

Get Goinfer (Linux):

wget https://github.com/synw/goinfer/releases/download/0.8.0/goinfer 
# or compile from source cf docs

Create a models directory somewhere and download a gguf model:

mkdir models
cd models
wget https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGUF/resolve/main/codellama-7b-instruct.Q4_K_M.gguf

Configure Goinfer:

mkdir tasks # internally required
./goinfer -localconf

Edit the generated goinfer.config.json file to add the absolute path to your models directory

Now clone https://github.com/synw/wingman and install it, then run code . and press F5. Configure the settings with these:

• OpenAi base url: http://localhost:5143
• OpenAi model: codellama-7b-instruct.Q4_K_M.gguf

Now set the api key with the one in the goinfer.config.json and it should work after a restart

Note: this setup is for example/convenience as it will not give good results with the base Codellama instruct model (waiting for some efficient fine tunes). I had much better results with even smaller models but it's easy to run as the gguf file is directly available for download.

[synw]

This provider is now deprecated