search

bigcode-project / starcoder.cpp

C++ implementation for 💫StarCoder
442 stars 36 forks source link

Servermode for working as endpoint for VSCode Addon "HF Code Autocomplete" #22

Open VfBfoerst opened 1 year ago

VfBfoerst commented 1 year ago

Hi,
is there already a feature to start this in something like a Server-Mode? I want to use starcoder as an endpoint for the VSCode-Addon HF Code Autocomplete and also later for a chat interface like huggingchat.

fwxfasp commented 1 year ago

hi, don't want to open a separate issue for this:

I wonder if this project still is in active development the last commit was 1 month ago, so I'm not sure if working with the cpp implementation makes any sense in general. also the communication in the issues doesn't seem to be too active. (no offense at all, I'm just curious 😄)

matthoffner commented 1 year ago

Hi, I have a FastAPI setup to demo a ggml Starcoder using ctransformers (https://github.com/marella/ctransformers):

https://huggingface.co/spaces/matthoffner/starchat-ggml UI chat demo: https://huggingface.co/spaces/matthoffner/starchat-ui

Not sure if it works with the HF VScode extension, will look into that!