search

zmedelis / bosquet

Tooling to build LLM applications: prompt templating and composition, agents, LLM memory, and other instruments for builders of AI applications.
https://zmedelis.github.io/bosquet/
Eclipse Public License 1.0
280 stars 19 forks source link

Question: Can I connect to an external service in the middle of the prompt chain? #27

Closed ronnyli closed 1 year ago

ronnyli commented 1 year ago

I'd like to get a prompt response from an LLM, send it to an API, then send the API response back into the LLM (and possibly repeat). Is that possible with Bosquet?

For example:

  1. First prompt: "You are a playwright, write a synopsis for a play"
  2. Call out to some external service, for example mechanical turk, to collect opinions about the synopsis
  3. Second prompt: "Perform sentiment analysis on the following responses: {{ EXTERNAL SERVICE RESPONSE }}"
  4. (back to 1): "Adjust your synopsis based on the feedback"
zmedelis commented 1 year ago

It is possible via agents https://github.com/zmedelis/bosquet#agents Thought support for that is very basic, with only calls to Wikipedia available. You'd need to extend the Tool protocol with Mechanical Turk implementation https://github.com/zmedelis/bosquet/blob/main/src/bosquet/agent/tool.clj

v0.4 release, sometime in early autumn, is planned to have extended support for this. I'll keep in mind the case you describe to add it in.

ronnyli commented 1 year ago

Very cool! And implementing a custom agent looks very straightforward. Awesome!