To use PromptTrail in the server to have conversations with a lot of people. We need to implement save and load conversation status middle of the running template because you can't save the Python state for everyone in memory.
This seems trivial but actually, it is very difficult.
Here's why:
Current implementation of flow control such as LoopTemplate is based on recursion.
Every template has a render method and LoopTemplate calls render of the child templates.
This can be nested to arbitrary depth.
Every render method can yield some messages. We may want to think about to dump and load state when a message is yielded.
[ ] Think about using return instead yield?
Then, our problem is:
We have a tree of templates and have some local variables for every template we have recursively visited (such as index of LoopTemplate etc.)
How we should save the state and load the state?
There are some ways to address this:
Implement Stack and Make every template can be loaded and be idempotence w.r.t. a given stack.
So, this is actually a kind of VM we're building here.
It is possible to write ControlTemplate like this. (Actually, the current implementation is partially minded about stack.)
But, do users want to do this? It's a very complex idea and not intuitive.
This kind of writing is required for non-control templates.
Cache all of the actions and mock everything.
This is also possible but requires the user to use some decorator or something for nondeterministic action. This is also not ideal.
We can loosen the problem:
We may only need to dump when template rendering is end?
This is better because you can just skip the render process using logged conversation log (Skip render and update the state to the exact status of previous execution)
We may do not need dump entirely.
This can be possible because maybe most applications don't require user input. If only machines interact with agent, then why not just restart?
To use PromptTrail in the server to have conversations with a lot of people. We need to implement save and load conversation status middle of the running template because you can't save the Python state for everyone in memory.
This seems trivial but actually, it is very difficult.
Here's why:
Current implementation of flow control such as LoopTemplate is based on recursion.
rendermethod and LoopTemplate callsrenderof the child templates.rendermethod can yield some messages. We may want to think about to dump and load state when a message is yielded.Then, our problem is:
There are some ways to address this:
We can loosen the problem: