search

lmstudio-ai / lms

👾 LM Studio CLI
https://lms.dev
MIT License
1.7k stars 138 forks source link

Statefullness | Persistence #48

Open nPHYN1T3 opened 4 months ago

nPHYN1T3 commented 4 months ago

This would be better as a discussion post but discussions are not enabled.

I was wondering if there is a way to enable statefullness | persistence? Without some type of "long term memory" this seems more like a cool party trick than anything of use. Having to rehash previously discussed issues/topics is a huge waste of time and processing power when trying to use a model on a project over the course of more than a day...unless you have a dedicated 24/7 machine going to simply never lose the original session...and some hella UPS's in case of a power event. ;p

Is there a way to do this? Am I missing something? Is this a future looking feature not implemented yet? Am I doing it wrong? Is this a trick to make the Ai forget all the horrible things you told it so when they robot uprising happens they don't come for you because they don't remember how abusive you were?

ugurlu2001 commented 3 months ago

Same issue on my side too.

Is there an easy way to remember previous conversations on LLMSTUDIO environment? Or is that possible?

nPHYN1T3 commented 3 months ago

I'm not sure if this is technically the same issue but another thing about the (lack of) persistence is simply there is no real effort by the model to "stay in the now" much less what's past. By this I mean it will give you some BS and you will correct it, it will apologize and spit out the same BS. So it doesn't even know it's already given you a wrong answer and just keeps giving the same BS in a loop. (Yes some models are better than others but they all have this generalized issue to some degree.) So there is a lack of persistence even in the current session.

So it's hard to even use "now" much less come back to something the next day. You're always spinning your wheels trying to get the model to remember or tweak sentences to get back to yesterdays understanding or to get it to acknowledge when something is false or unhelpful and revise its info. Never mind I can't get any prompt to make it cut the crap. Every answer is laced with so much BS fluff even if it's correct it's like a bad high school paper trying to pad everything to a word limit...sooo frustrating. Wait I take that analogy back. It's like talking to an "old" person who just wants to talk. You: "Yo model, 2+2 is what?" Model: Mathematics goes back to when Ronald Reagan wrote and produced StarWars. Back when Camels roamed the Earth and the Sun was just a toy..." It's just a waste of computation all around to get anywhere. Ai/ML? More like digital groundhog day crossed with drunk Grandpa.

Perhaps I'm just expecting too much and this is just the state of most models and their underlying technology so feel free to let me know if this is simply the case.