LM Studio partial support

DiAifU commented 6 months ago

What happened?

Hi,

LM Studio, with a locally running model, is working by using OpenAI with a Preset of Ollama or Lama Cpp and changing the port to the default LM Studio one (1234). The prompts are being generated but always end up with the error "Unknown API response. Code: 200, Body:".

Could you please tell me how to fix this or add support for responses being generated by LM Studio ?

Thanks for the great plugin !

Nicolas

Relevant log output or stack trace

Unknown API response. Code: 200, Body:

Steps to reproduce

Run LM Studio with a model locally
Start a server in LM Studio
Configure CodeGPT to target that server (OpenAI + Ollama or Lama Cpp Preset + change port to 1234)

CodeGPT version

2.4.0

Operating System

Windows

carlrobertoh commented 6 months ago

Thank you for reporting!

The error appears to be related to the OkHttp library and how it processes the event streams. For some reason, LM Studio doesn't seem to append empty newline at the end of the final response, causing OkHttp to fail. I am not yet sure what the fix is.

raivisdejus commented 6 months ago

Maybe @lmstudio-ai can sort this out on their end...

carlrobertoh commented 6 months ago

https://github.com/langchain4j/langchain4j/issues/670 related issue.

The error java.lang.IllegalArgumentException: byteCount < 0: -1 can be reproduced by removing the empty newlines from the mocked response: LocalCallbackServer.java#L111

lucacri commented 4 months ago

Same problem here, any suggestions on how to temporarily fix it?

carlrobertoh / CodeGPT