search

atisharma / chasm_engine

CHAracter State Management - a generative text adventure (engine)
https://chasm.run
GNU Affero General Public License v3.0
61 stars 5 forks source link

/help is calling the LLM #5

Closed kadogo closed 1 month ago

kadogo commented 1 year ago

Hello @atisharma

I noticed that when I called /help command, it was calling the LLM. Is there any reason for that?

I didn't really read the full code yet because I don't know hy so I will read about it later. If there is no reason to use the LLM if /help is called, I think it would be better to parse it before. I don't know whether it's straightforward to do or not, when I understand hy a bit better, I will maybe give it a try.

Cheers

atisharma commented 1 year ago

It's not directly calling the LLM. What's happening is that the background thread is developing the world (spawning items, characters etc). There's a lot of this in a new world and it can take a long while to settle down.

kadogo commented 1 year ago

Hello, So the best way would to have a way that calling direct commands are not blocking by the world generation? I don't know if it's something that is achievable with the actual implementation.

atisharma commented 1 year ago

For some reason my email reply didn't make it here, so I copy:

The're not directly blocking, because the world generation happens in a separate thread. However, if you only have a single LLM server defined, the calls to the LLM API will block due to contention (the API server will be busy).

I usually use either the chatgpt API or two 13B models served simultaneously in which case there is no contention.

kadogo commented 1 year ago

Alright, so even if it doesn't use the LLM for the /help command, it is still needed to have one other API server available.

I will give it a try when I have a bit of time, thanks.

atisharma commented 1 year ago

If you want to just try the game without setting up a server, you can have a look at https://chasm.run