search

mlc-ai / web-llm

High-performance In-browser LLM Inference Engine
https://webllm.mlc.ai
Apache License 2.0
13.66k stars 878 forks source link

vercel/ai provider integration #551

Open louis030195 opened 3 months ago

louis030195 commented 3 months ago

https://github.com/vercel/ai is the best lib to build amazing AI experiences with tools, good user feedback, etc.

they have a concept of provider which is way to integrate ai framework / api in their lib https://sdk.vercel.ai/providers/ai-sdk-providers

i did a dirty provider once, cant find it anymore but this is some hook i did before:

https://github.com/llm-edge/hal-051224/blob/main/lib/hooks/use-web-llm-chat-engine.tsx

would be amazing to have a clean web-llm provider for vercel/ai

Neet-Nestor commented 3 months ago

Thanks for providing this idea! We will have a look and check whether we could integrate web-llm with it.