LiteLLM

[michaelsharpe]

What Changed:

• LiteLLM added with JsPyBridge
• Adds generate text node
• Adds streamMessage node for completion streaming
• chat window in spell composer no supports token streaming

How to test:

How can the reviewer test your changes and validate that they work as described?

Additional information:

Any other information that might be useful while reviewing.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
magick-client	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 5, 2024 6:07pm
magick-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 5, 2024 6:07pm

[michaelsharpe]

There are still a few more things we need to do with this:

• improve error handling when LiteLLM throws errors and surface those errors into the IDE
• Maybe merge streaming and non streaming mode so streaming is always happening, and at the end we just return the complete text and trigger done
• Add a <> token when streaming finishes as well
• properly document requirements for spinning up the repo with Python stuff in the Readme
• Ensure this deploys to heroku properly
• Set up API keys in secrets manager for display to user
• Filter choices of models based on what API keys have been provided.