Closed cproctor closed 1 week ago
I have created an initial implementation, creating an app called text_generation
instead of llm
. (My thinking for the name is that we should focus on what it does rather than the technology on which it's implemented. However, this initial version is not adequate; I'll keep this issue open until it's improved.
Whichever features we choose to implement, we will need a backend means of calling out to a LLM. Instead of hard-coding OpenAI's APIs into the project, I would like a vendor-neutral API, which for now will wrap around OpenAI, but which could be extended to other LLM APIs in the future.
llm
within the Unfold Studio project.unfold_studio.llm
which provides an interface to LLM estimation. Probably this should be a class which can be initialized with settings.unfold_studio.llm.models
, create a data model to log LLM queries. Should contain query, response, API name, timestamp, etc.cache_queries
is enabled, the API should check whether there is already a saved query response in the database.base_settings.py
parameterizing the LLM API (e.g. OpenAI API keys; a boolcache_queries
setting, etc. ).unfold_studio.llm
which only searches for cached responses, otherwise returns a constant string.Once this layer is in place, we'll write a route which the frontend can use to query the LLM. The front-end runtime will need to make dynamic calls to LLM for content during story playthrough.