Open RihanArfan opened 1 week ago
Run & review this pull request in StackBlitz Codeflow.
commit: 005ecaf
``` npm i https://pkg.pr.new/nuxt-hub/core/@nuxthub/core@173 ```
- [default](https://pkg.pr.new/template/7f1d68c5-d45f-476f-a54b-86ccd1f990b7)
@Atinux Where would you like the docs for AI? Under "Server"?
Well this is a very good question, I think we should create a new section, any name ideas?
I think ":sparkles: AI" with AI and Vectorize within. Other future features like Queues would fit nicely under Server.
Btw is there a reason we're importing types from @nuxt-hub/core
existing runtime utils? I copied this format, but I get errors until directly importing the type.
Did you run npm run prepack
and restarted VS COde @RihanArfan ?
Closes #84, Related https://github.com/nuxt-hub/core/tree/feat/ai, Related #174
Adds Workers AI support to Nuxt Hub.
Current issues/notes
Authentication
wrangler requires authentication to use Workers AI bindings locally as the models will still run on Cloudflare and incur usage. As
nitro-cloudflare-dev
doesn't pass wrangler's console output or show a URL to authenticate, unless you've used wrangler directly with Workers bindings locally before, you get errorError: Not logged in.
.I've opened issue https://github.com/pi0/nitro-cloudflare-dev/issues/26 upstream to try and get this resolved. A Cloudflare employee has responded and hopefully we get an ETA in a few days.
The current workaround is manually logging in one time system-wide using
npx wrangler login
.Alternative workaround is supporting AI through --remote via the proxy so it just runs directly from the Cloudflare account.
If you have access to multiple accounts, see docs below.
Streaming
Output appears mangled. See streaming code snippet below. This may be due to the old model I tried, or from using it via local development in Wrangler.
Details
**Response:** ``` data: {"response":" ","p":"abcdefghijklmnopqr"} data: {"response":" Ah","p":"abcdefghijklmnopqrstuvwxyz0123456789abcdefgh"} data: {"response":",","p":"abcd"} data: {"response":" a","p":"abcdefghijklmnopqrstu"} data: {"response":" curious","p":"abcdefghijklmnopqrstuvwxyz0123456"} data: {"response":" question","p":"abcdefghijklmno"} ... data: [DONE] ``` **⚠️ Expected response should be this format but it's not.** ``` data: {"response":"New"} data: {"response":" York"} data: {"response":" is"} data: {"response":" located"} data: {"response":" in"} data: {"response":" the"} ... data: [DONE] ```