nuxt-hub / core

Build full-stack applications with Nuxt on CloudFlare, with zero configuration.
https://hub.nuxt.com
Apache License 2.0
802 stars 35 forks source link

feat: add ai #173

Open RihanArfan opened 1 week ago

RihanArfan commented 1 week ago

Closes #84, Related https://github.com/nuxt-hub/core/tree/feat/ai, Related #174

Adds Workers AI support to Nuxt Hub.

Current issues/notes

Authentication

wrangler requires authentication to use Workers AI bindings locally as the models will still run on Cloudflare and incur usage. As nitro-cloudflare-dev doesn't pass wrangler's console output or show a URL to authenticate, unless you've used wrangler directly with Workers bindings locally before, you get error Error: Not logged in..

I've opened issue https://github.com/pi0/nitro-cloudflare-dev/issues/26 upstream to try and get this resolved. A Cloudflare employee has responded and hopefully we get an ETA in a few days.

The current workaround is manually logging in one time system-wide using npx wrangler login.

Alternative workaround is supporting AI through --remote via the proxy so it just runs directly from the Cloudflare account.

If you have access to multiple accounts, see docs below.

Streaming

Output appears mangled. See streaming code snippet below. This may be due to the old model I tried, or from using it via local development in Wrangler.

Details **Response:** ``` data: {"response":" ","p":"abcdefghijklmnopqr"} data: {"response":" Ah","p":"abcdefghijklmnopqrstuvwxyz0123456789abcdefgh"} data: {"response":",","p":"abcd"} data: {"response":" a","p":"abcdefghijklmnopqrstu"} data: {"response":" curious","p":"abcdefghijklmnopqrstuvwxyz0123456"} data: {"response":" question","p":"abcdefghijklmno"} ... data: [DONE] ``` **⚠️ Expected response should be this format but it's not.** ``` data: {"response":"New"} data: {"response":" York"} data: {"response":" is"} data: {"response":" located"} data: {"response":" in"} data: {"response":" the"} ... data: [DONE] ```
pkg-pr-new[bot] commented 1 week ago

Review PR in StackBlitz Codeflow Run & review this pull request in StackBlitz Codeflow.

commit: 005ecaf

@nuxthub/core

``` npm i https://pkg.pr.new/nuxt-hub/core/@nuxthub/core@173 ```


templates

- [default](https://pkg.pr.new/template/7f1d68c5-d45f-476f-a54b-86ccd1f990b7)

RihanArfan commented 1 week ago

@Atinux Where would you like the docs for AI? Under "Server"?

Atinux commented 1 week ago

Well this is a very good question, I think we should create a new section, any name ideas?

RihanArfan commented 1 week ago

I think ":sparkles: AI" with AI and Vectorize within. Other future features like Queues would fit nicely under Server.

Btw is there a reason we're importing types from @nuxt-hub/core existing runtime utils? I copied this format, but I get errors until directly importing the type.

image

image

image

Atinux commented 6 days ago

Did you run npm run prepack and restarted VS COde @RihanArfan ?