Closed NK-Aero closed 8 months ago
I'm unfamiliar with LM Studio, but I just downloaded it and started a local inference server running llama 2 chat 7B q4_0 ggml
. Using the OpenAI provider in Wingman, I changed the URL to http://localhost:1234/v1/chat/completions
and things seem to just work (with the exception of recognizing the end of the completion stream, which may just be unique to LM Studio, but a bug none-the-less).
Can you provide some more information regarding your setup?
KoboldCpp support is almost finished - still porting this functionality from the previous major version.
Thank you , it is working now. i was using "http://localhost:1234/v1". now i changed to "http://localhost:1234/v1/chat/completions" . it is working.
I was using Wingman 1.3.8 preview, it was excellent. Now, it upgraded well. Thank you.
with the exception of recognizing the end of the completion stream, which may just be unique to LM Studio, but a bug none-the-less
This is fixed now and the completion stream resoled by LM Studio should correctly end the response. Pushing a release with this bug fix now.
Thank you , it is working now. i was using "http://localhost:1234/v1". now i changed to "http://localhost:1234/v1/chat/completions" . it is working.
I was using Wingman 1.3.8 preview, it was excellent. Now, it upgraded well. Thank you.
Excellent! Glad it's working. Have fun!
Wingman v2.0.8 , Local model is not supporting. LM Studio, Koboldcpp URL are not working, please provide documentation or tutorial or video about enabling local models. Thank you