voidKandy / espionox

Use espionox to streamline making LLM applications in Rust
https://crates.io/crates/espionox
10 stars 0 forks source link

Finish v0 refactor #23

Closed voidKandy closed 7 months ago

voidKandy commented 7 months ago

Before anything else the v0 refactor should be finished

voidKandy commented 7 months ago

Commit #1e6f6583d2eb3fc880beec9b97b38a17ce170934 Is a very hefty one. Agents now require being added to an Environment in order to get completions. All web requests and such are made on the environment thread. Eventually LTM will be moved to environment. Basic logic is worked out, but the API is underdeveloped

voidKandy commented 7 months ago

Finally figured out a working implementation of the new environment system. As said in my previous comment, agents now required a connection to an environment to get completions. Environment contains a thread and a dispatch for doing things on that thread. Instead of developers directly prompting agents, they now have to get a handle to an agent within the environment with the get_agent_handle() method. Which, given an ID, returns a handle to an inner agent. The handle contains methods like request_completion(), which sends a message to dispatch to request a completion. The resulting completion can be accessed by joining the environment thread by calling finalize_dispatch() and then grabbing the completion message from the response stack via get_responses_stack. Obviously, the complexity of the API has increased significantly, but I think the new system will actually make implementing things like CoT, ToT, and the old 'Observer' system much easier

voidKandy commented 7 months ago

v0_refactor is ready to be merged with master. All previous features but Database integrations are working. Database will need to be completely redone and will be released in 0.2