useChat breaks custom SWR fetcher implementation

[LeakedDave]

Description

I've got a pretty simple fetcher function for a custom Cache implementation in my app, useChat is totally broken with it since it is sending tons of weird requests where url is an Array. Is this lib not compatible with SWR

import useSWR, { SWRConfig, useSWRConfig } from "swr"
import { useSession } from "@supabase/auth-helpers-react"
import { useCacheProvider } from "@piotr-cz/swr-idb-cache"

/** Hook for clearing cache */
export const useClearCache = () => {
    const { cache } = useSWRConfig()

    const clearCache = () => {
        for (let key of cache.keys()) cache.delete(key)
    }

    return clearCache
}

/**
 * Overrides SWR isLoading to true while provider is loading
 * @param {string} query - The query to fetch
 * @param {object} config - The options for the fetch
 * @returns {import("swr").SWRResponse} - The SWR response
 */
export const useCache = (query, config) => {
    const { provider } = useSWRConfig()
    const swr = useSWR(provider ? query : null, config)

    return { ...swr, isLoading: swr.isLoading || !provider }
}

/** Integrates SWR with IndexedDB to persist the cache */
export default function CacheProvider({ children }) {
    const session = useSession()
    const cacheProvider = useCacheProvider({
        dbName: 'swr-cache',
        storeName: 'swr-cache',
    })

    const fetcher = async (url) => {
        if (Array.isArray(url)) {
            console.log("Array URL", url)
            return
        }

        const headers = {}
        let basePath = ""

        // Use base URL for export
        if (process.env.NEXT_PUBLIC_IS_EXPORT == "1") {
            // Pass session access token
            if (session?.access_token) {
                headers['Authorization'] = `Bearer ${session.access_token}`;
            }

            if (!url.startsWith("http")) {
                basePath = process.env.NEXT_PUBLIC_BASE_URL
            }
        }

        const res = await fetch(basePath + url, { headers })
        const json = await res.json()
        if (res.ok) return json
    }

    return (
        <SWRConfig value={{
            fetcher,
            provider: cacheProvider,
            // focusThrottleInterval: 30000,
            // dedupingInterval: 30000
        }}>
            {children}
        </SWRConfig>
    )
}

Array URL (2) [':Rd7hcm:', 'loading']
cache-provider.js:40 Array URL (2) [':Rd7hcm:', 'streamData']
cache-provider.js:40 Array URL (2) [Array(2), 'loading']
cache-provider.js:40 Array URL (2) [Array(2), 'streamData']
cache-provider.js:40 Array URL (2) [Array(2), 'error']
cache-provider.js:40 Array URL (2) ['/api/completion', ':Rd7hcm:']
cache-provider.js:40 Array URL (2) [Array(2), 'messages']0: (2) ['/api/chat', ':Rl7hcm:']1: "messages"length: 2[[Prototype]]: Array(0)
cache-provider.js:40 Array URL (2) ['545d6e2d-9841-4edf-9555-c0c05d8d34ba', 'loading']0: "545d6e2d-9841-4edf-9555-c0c05d8d34ba"1: "loading"length: 2[[Prototype]]: Array(0)
cache-provider.js:40 Array URL (2) ['545d6e2d-9841-4edf-9555-c0c05d8d34ba', 'streamData']0: "545d6e2d-9841-4edf-9555-c0c05d8d34ba"1: "streamData"length: 2[[Prototype]]: Array(0)
cache-provider.js:40 Array URL (2) [Array(2), 'loading']0: (2) ['/api/chat', '545d6e2d-9841-4edf-9555-c0c05d8d34ba']1: "loading"length: 2[[Prototype]]: Array(0)
cache-provider.js:40 Array URL (2) [Array(2), 'streamData']0: (2) ['/api/chat', '545d6e2d-9841-4edf-9555-c0c05d8d34ba']1: "streamData"length: 2[[Prototype]]: Array(0)
cache-provider.js:40 Array URL (2) [Array(2), 'error']0: (2) ['/api/chat', '545d6e2d-9841-4edf-9555-c0c05d8d34ba']1: "error"length: 2[[Prototype]]: Array(0)

This broke when I updated my project to the latest version of the React AI sdk

[LeakedDave]

I'm pretty sure the solution is going to be to put custom fetchers and cache providers in the lib itself so it doesn't affect the main application's SWR Configuration. useChat should be standalone and not have any interference with the parent app's SWR.

For now I will lock my ai to v3.2.19 where everything works OK. Would be nice to get up to date though.

[lgrammel]

Is 3.2.20 the specific version where it broke for you?

[LeakedDave]

Not sure, I've had it at 3.2.19 for a long time and was just updating all my packages and ran into this issue. I can try bumping it version by version to find the exact one but it's definitely related to taking over the SWR Config. I wrap my NextJS _app.js with the provider above. useChat does work when I disable the custom fetcher, but this breaks my entire project.

[LeakedDave]

I think the fix for this lib to function side by side with custom SWR configurations is to supply and override with your own fetcher and provider every time you call "useSWR". In fact I would override all of the useSWR calls with an entire SWR Config with default values, so it doesn't interfere with the parent project's SWR Config. The offending files are below

packages/react/src/use-chat.ts

packages/react/src/use-completion.ts

packages/react/src/use-object.ts

[LeakedDave]

In Svelte I see you guys have this custom fetcher, I'm assuming the React side needs something similar

fetcher: () => store[key] || initialCompletion,

[lukasver]

Experiencing the same issue. My custom SWR config gets picked up by the useSWR hook in useChat with unexpected & unwanted behaviour, like doing subsequents GET requests to the keyed endpoint. Going back to 3.2.19 from 3.4.29 stoped the issue.

Also tried to override my custom swr fetcher by using a dedicated SWRConfigs for the useChat hook but did not work.

[LeakedDave]

Experiencing the same issue. My custom SWR config gets picked up by the useSWR hook in useChat with unexpected & unwanted behaviour, like doing subsequents GET requests to the keyed endpoint.

Going back to 3.2.19 from 3.4.29 stoped the issue.

Also tried to override my custom swr fetcher by using a dedicated SWRConfigs for the useChat hook but did not work.

My temporary fix to use the latest version was to simply create a wrapper for useSWR and put the custom fetcher in that hook instead of in the SWRConfig, and update all references in the project to use this custom hook. It's not the cleanest fix but it does prevent the unwanted requests coming from AI SDK.

[lgrammel]

packages/react/src/use-chat.ts

packages/react/src/use-completion.ts

packages/react/src/use-object.ts

@LeakedDave is this happening just with useChat or can you also reproduce with useCompletion?

[lgrammel]

Just looked through the changes and there have been no SWR-related changes since 3.2.19. Would need to bisect the versions to find where it's breaking.

[LeakedDave]

packages/react/src/use-chat.ts

packages/react/src/use-completion.ts

packages/react/src/use-object.ts

@LeakedDave is this happening just with useChat or can you also reproduce with useCompletion?

Both useChat and useCompletion are calling custom fetchers from a parent SWRConfig. I don't know what version it was introduced but I did see in the source code that you guys useSWR with array keys and custom mutates to store data. I think the solution is to simply add a blank fetcher to the useSWR calls that the sdk makes with these array keys

[LeakedDave]

For example

const { data: messages, mutate } = useSWR<Message[]>( [chatKey, 'messages'], null, { fallbackData: initialMessages ?? initialMessagesFallback }, );

In the object with fallbackData, this needs a fetcher: () => null

Like a blank function to ensure it doesn't call the parent's function. I think in the Vue sdk you guys are handling this a little differently.

I don't think the AI sdk needs a fetcher at all, it appears that the data is manually mutated in with mutate(object, false)