search

lmstudio-ai / lms

LM Studio CLI. Written in TypeScript/Node
https://lms.dev
MIT License
1.13k stars 87 forks source link

Running in headless mode #7

Open donvito opened 2 months ago

donvito commented 2 months ago

Is is possible to run LMStudio in headless mode when running the API? Use case is running it in a Linux instance without a UI.

ryan-the-crayon commented 2 months ago

Unfortunately, we currently do not officially support running LM Studio in headless mode. It is something we are currently investigating, but if you have some insights, we are all ears.

That said, there was some discussion on the topic yesterday on our discord's #dev-chat channel, and someone was able to run it flawlessly. You might be interested to take a look there.

https://discord.gg/pwQWNhmQTY

POD319 commented 2 months ago

Headless mode is essentially running without the GUI, right? I was about to make a separate post but I think this is my same concern. I was discouraged that there was no way to run LM Studio without keeping the window open at all times. So, I figured LMS was made specifically for that reason -- to run without the GUI. However, every command I am trying requires it to open the GUI.

Is there a least a way to hide the GUI to the toolbar? Having a program window required to be opened seems like a rare thing nowadays.

I think someone mentioned Oolama elsewhere -- I like LM Studio for having the GUI option, but running how Oolama can would be fantastic.

I think my overall question is -- what is the use case for LMS in general? I think I'm missing something...

ryan-the-crayon commented 2 months ago

@POD319 Enabling running LM Studio without GUI is definitely our goal. However, we are still investigating on how to do it.

If your are curious, more specifically:

We already support running in tray on Mac OS (without window).

For Windows, we are still investigating how to make sure the user is aware that LM Studio is still running when it is only in tray, especially when Windows might decide to hide it in the secondary menu.

For Linux, we need to figure out how to run the application in "true" headless mode, that is to start the application at all when there is no display. As you can see, there are some magical incantation on that discord channel, that we are not sure how stable they are.

In any case, we are very interested to hear your specific use cases that you would need headless mode for.

POD319 commented 2 months ago

@POD319 Enabling running LM Studio without GUI is definitely our goal. However, we are still investigating on how to do it.

For Windows, we are still investigating how to make sure the user is aware that LM Studio is still running when it is only in tray, especially when Windows might decide to hide it in the secondary menu.

In any case, we are very interested to hear your specific use cases that you would need headless mode for.

Appreciate the quick reply! Windows hiding those icons is quite annoying. I have that setting disabled, so I can always see what is running, but I know that is not the default.

For me, it's really just the minimalist approach. I like to be able to glance over to my toolbar and see what is running there. Open windows are typically something that I am actively working with. Often times I would leave LMStudio running and be working with the server running a model through AnythingLLM or a similar API call. Having the GUI window open just kind of made my taskbar 'messy' for me, and I found myself closing out the window often, accidentally killing the server in the process.

Looking forward to when this is implemented, thanks!

ruzzzz6312 commented 1 month ago

+1 waiting for headless implementation, thanks!