Cloudflare Vectorize

[RihanArfan]

Is your feature request related to a problem? Please describe. I'd like to use Cloudflare Vectorize (database for storing vectors) alongside Workers AI (#173).

Describe the solution you'd like Vectorize is similar to D1 where it's a database. It's implementation would look similar to D1 in Nuxt Hub.

Describe alternatives you've considered Manually adding a binding and directly using the API (#113)

Additional context

• https://developers.cloudflare.com/vectorize/reference/client-api/
• https://developers.cloudflare.com/vectorize/reference/metadata-filtering/

Happy to contribute a simple PR into @nuxt-hub/core to add hubVectorize() now but it wouldn't include proxying remote or devtools viewer though.

Also, should it be called hubVectorize() or should it have a different name like hubVectorDatabase()?

[atinux]

Thank your for creating the issue @RihanArfan

I think hubVectorize() makes the more sense, feel free to open a PR to start the work 😊

[RihanArfan]

While implementing it, I've ran into things which require big decisions to be made. Also moving this here instead of the PR description so it's clearer.

Multiple bindings:

• Vectorize supports namespaces in indexes (basically like tables) but all models need to be the same dimensions/metric
• Any application using text embedding models of different models (small, base, large) simultaneously would need separate bindings due to different dimension sizes. I guess most people would probably want separate index per thing (products/reviews for e.g.)
• Cloudflare supports creating an index via Wrangler or via the API.

Remote bindings:

• Wrangler doesn't currently support Vectorize with local bindings, so --remote is necessary to use Vectorize via the deployed application. See https://github.com/cloudflare/workers-sdk/issues/4360.
• Therefore, even for local development a remote database is needed. This approch of remote bindings during dev is already done by hubAI().
• How many indexes exist for local development?
  • One (local-development) - Shared among all developers in the project locally developing
  • One per developer (local-development-rihan-arfan) - Each developer gets a separate index
  • Multiple per developer (local-dev-abcde) - Developer can select which "local" dev index they want, and use the same across developers
  • nuxthub vectorize dev <binding> <create/list/delete/reset>
  • nuxthub vectorize dev products list (shows all development indexes for products binding, including who made and optional description)
  • nuxthub vectorize dev products use asdfg

Alternatively we just wait for Cloudflare to support local Vectorize bindings in wrangler to save massive complexity and overengineering 😄

Resetting indexes for local developent:

• Currently, you can only delete all contents in an index by recreating the index itself and redeploying (compared to getall and mass deleting).
• For local development, there needs to be a way to easily recreate the remote database (either to wipe data or to change dimensions/metric.
  • Could allow resetting via Nuxt devtools, a cli command, dashboard or all
  • nuxthub vectorize reset-dev products (if one local-dev index exists per binding)
  • nuxthub vectorize dev products reset (if multiple local dev indexes possible)

Wrangler would probably have a way to do this once local development is supported.

Approaches for managing indexes:

How Vectorize indexes should be managed using NuxtHub across different environments, etc.. Unlike other features like databases, a vectorize index needs options provided during creation based on the text embeddings model a user plans to use. Vectorize indexes also require specifying metadata indexes upfront if you want to use metadata filtering.

Any of these options would be used using hubVectorize(<binding>):

const vectorize = hubVectorize("products")

Here are some different options I've thought of to handle it.

Option A

• User creates index manually and lets NuxtHub know what the name+environment, so NuxtHub can add the binding
• Users would need to login to Wrangler themselves, unless we recreate functionality into NuxtHub's CLI via existing login.

pnpx wrangler vectorize create foo-ecommerce-products --dimensions=768 --metric=cosine
pnpx wrangler vectorize create foo-ecommerce-products-preview --dimensions=768 --metric=cosine
pnpx wrangler vectorize create foo-ecommerce-products-development --dimensions=768 --metric=cosine

export default defineNuxtConfig({
  hub: {
    // user needs to create metadata indexes via cli
    vectorize: {
      products: {
        production: 'foo-ecommerce-products',
        preview: 'foo-ecommerce-products-preview',
        development: 'foo-ecommerce-products-development'
      },
      reviews: {
        production: 'your-vectorize-id',
        preview: 'your-vectorize-id',
        development: 'your-vectorize-id'
      }
    },
  }
})

Option B

Specifying index details (dimensions, metric) via nuxt.config.ts. This approach needs extending to add metadata indexes, which are necessary to filter vectors via metadata.

    vectorize: {
      // nuxthub handled creation of the index across environments
      products: {
        metric: 'cosine',
        dimensions: 768,
      }
      // use own vectorize indexes
      reviews: {
        production: 'your-vectorize-id',
        preview: 'your-vectorize-id',
        development: 'your-vectorize-id'
      }
    }

DX might be confusing as changing the config probably shouldn't result in automatically recreating production index to prevent accidental data loss. A potential fix is keeping the old Vectorize index but simply pointing the binding to a new index.

• foo-ecommerce-products (dimensions 384)
• foo-ecommerce-products2 (dimensions 1024) <- PRODUCTS binding links to this

Option C

Create, reset and delete indexes via a NuxtHub CLI and/or dashboard. All handled via CLI and backend rather than nuxt.config.ts. On start of dev server, Nuxt checks what indexes are available. This approach allows manually managing what indexes exist on each environment, including using existing indexes.

export default defineNuxtConfig({
  hub: {
    database: true,
    vectorize: true,
  },
});

$ # vectorize specific - future multi-bindings could have individual things
$ nuxthub vectorize create products --dimensions=768 --metric=cosine
# Done! Binding: PRODUCTS Index: foo-ecommerce-products, foo-ecommerce-products-preview, foo-ecommerce-local
# Use via `useVectorize("products")`

$ nuxthub vectorize list
# Vectorize indexes associated with "foo-ecommerce":
# [PRODUCTS]: # hubVectorize('products')
# foo-ecommerce-products         | dimensions: 768 | metrics: cosine
# foo-ecommerce-products-preview | dimensions: 768 | metrics: cosine
# foo-ecommerce-products-local   | dimensions: 768 | metrics: cosine
#
# [REVIEWS]:
# foo-ecommerce-products-local   | dimensions: 768 | metrics: cosine
#
# [KNOWLEDGEBASE]:
# support-system-articles        | dimensions: 768 | metrics: cosine
#
# ----------
# Create new index:
# nuxthub vectorize create <name> [--dimensions=<int>] [--metric=<string>] [--environments=<string=all>]
#
# Link an existing index:
# nuxthub vectorize link-existing-index support --index=support-system-articles-preview --environments=preview# 
#
# Create a metadata index for an index:
# nuxthub vectorize create-metadata-index products --environments=all --property-name=streaming_platform --type=string
#
# Recreate local-development index (other environments would need deleting and recreating explicitly)
# nuxthub vectorize reset-dev products