Closed skprasadu closed 1 year ago
I was not familiar with this library. I played a bit with it and here is how it can integrate with Chainlit at first glance:
import chainlit as cl
import guidance
from guidance._program import Log
guidance.llm = guidance.llms.OpenAI(model="gpt-3.5-turbo")
class ChainlitLog(Log):
def append(self, entry):
super().append(entry)
print(entry)
is_end = entry["type"] == "end"
is_assistant = entry["name"] == "assistant"
if is_end and is_assistant:
cl.run_sync(cl.Message(content=entry["new_prefix"]).send())
@cl.on_message
async def main(message: str):
program = guidance(
"""
{{#system~}}
You are a helpful assistant
{{~/system}}
{{~#geneach 'conversation' stop=False}}
{{#user~}}
{{set 'this.user_text' (await 'user_text') hidden=False}}
{{~/user}}
{{#assistant~}}
{{gen 'this.ai_text' temperature=0 max_tokens=300}}
{{~/assistant}}
{{~/geneach}}""",
)
program(user_text=message, log=ChainlitLog())
This is very minimal and we have to explore what exactly is available in the entry
dict. Also some string cleaning will be necessary.
Microsoft Guidance seem to be a powerful prompt engineering tool. Is there a working example of how chainlit works with this.
I came up with an example, but not sure if this is a best practice.
This works great. But I get below prompt in my frontend. Not sure if there is a way to format this.
Any help will be appreciated.