search

zmedelis / bosquet

Tooling to build LLM applications: prompt templating and composition, agents, LLM memory, and other instruments for builders of AI applications.
https://zmedelis.github.io/bosquet/
Eclipse Public License 1.0
280 stars 19 forks source link

allow caching calls to `complete` #19

Closed behrica closed 1 year ago

behrica commented 1 year ago

This PR allows to cache the calls to "complete". It is optional and gets configured by adding a implementation of a clojure.core.cache in the params like this:

(def azure-open-ai-config
    {:model "text-davinci-003"
     :impl openai/complete-azure-openai
     :cache  (clojure.core.cache.wrapped/fifo-cache-factory {} :threshold 1)
     :api-key "xxxx"
     :api-endpoint "https://yyyyy.openai.azure.com/"})

The code depends on PR #17 and would then fix #11