zmedelis / bosquet

Tooling to build LLM applications: prompt templating and composition, agents, LLM memory, and other instruments for builders of AI applications.
https://zmedelis.github.io/bosquet/
Eclipse Public License 1.0
280 stars 19 forks source link

configuration of llms not consistent #33

Closed behrica closed 1 year ago

behrica commented 1 year ago

Between gen/complete-template:

(def synopsis
  (gen/complete-template
    synopsis-template
    {:title "Mr. X" :genre "crime"}
    {:play   :llm/openai
     :review :llm/cohere}))

and gen/complete

(def review (gen/complete play 
    {:title "Mr. X" :genre "crime"}
    {:synopsis {:llm-generate {:impl :openai :api-key "<my-key>"}}
     :review   {:llm-generate {:impl :openai :api-key "<my-key>"}}}))

Maybe we mix here the "new style" and the "old style". This relates as well to #30

So maybe the code in 0.3.6 is still in a transition to the new style, as introduced by the change to "Config management for LLM components"

So I will wait for your feedback, before looking deeper into #30. I think, that the code as in 0.3.6 cannot work for Azure with "gen/complete-template"

behrica commented 1 year ago

It took me a bit, but I finally realized that gen/complete-template has a subset of functionality then gen/complete, correct ? So all I can do with gen/complete-template can be done by using gen/complete. If this is true, I would suggest to remove gen/complete-template .This should make teh code simpler.

zmedelis commented 1 year ago

Added a fix for the complete-template configuration. Both complete and complete-template accept the last param with LLM service and model options (see comment section in bosquet.llm.generator)

  (complete-template
   "You are a playwright. Given the play's title and its genre write a synopsis for that play.
     Title: {{title}}
     Genre: {{genre}}
     Playwright: This is a synopsis for the above play: {% gen var-name=text %}"
   {:title "Mr. X" :genre "crime"}
   {:text {:bosquet.llm/service [:llm/openai :provider/openai]
           :bosquet.llm/model-parameters {:temperature 0 :model "gpt-4"}}})

I will be doing an iteration or two on this parameterization approach and thinking about more or less freezing it for v0.4.0

If this is true, I would suggest to remove gen/complete-template .This should make teh code simpler.

See the comment I added in the complete-template. There might be a benefit in having a slightly simpler version of the complete function, but you are probably right, not worth added complexity. Will consider removing it.

In any case, those are big interface changes. Need way better documentation. Will start work on that instead of dumping everything into Readme and notebooks.

zmedelis commented 1 year ago

Not to hold the other issues, closing it. Addressed in v0.3.7 release.