search

sashirestela / simple-openai

A Java library to use the OpenAI Api in the simplest possible way.
MIT License
170 stars 16 forks source link

Support for LMStudio? #172

Closed bertilmuth closed 2 months ago

bertilmuth commented 2 months ago

I'm having a hard time to send requests to LM Studio.

This is very odd, since I am able to send the same requests to LLamafile, with the same base URL: http://localhost:1234

I was able to debug CleverClient to see that the request is sent to LMStudio, but it never arrives there. It might be a firewall issue. But maybe somebody else has experienced similar problems?

sashirestela commented 2 months ago

Hi @bertilmuth, thanks for using simple-openai!

It would be very helpful if you provide your full request and the error message in simple-openai or cleverclient, please.

For some local restrictions I cannot install LMStudio or any similar software, sorry.

bertilmuth commented 2 months ago

Here's a screenshot of the position where the problem occurs (I think). Unfortunately, there is no error message. The thread just "hangs" and it seems the request is never received (i.e. the join waits forever). Thank you for your support.

image

Btw POST request headers are { {Authorization=[Bearer], Content-Type=[application/json]} } which means: I pass no token, since I use a local LLM. This shouldn't be a problem though, since sending a POST request with Postman seems to work with these headers.

bertilmuth commented 2 months ago

Closing, since it seems to be a Firewall issue

sashirestela commented 2 weeks ago

@bertilmuth Were you able to run LMStudio locally? Could you tell us what the issue was and how you solved it, please? Thanks in advance for any response.

sashirestela commented 2 weeks ago

Nevermind, the user @danielsawan was able to solve this in a great way:

https://github.com/sashirestela/simple-openai/issues/198#issuecomment-2422713023

Thanks anyway!