[Bug]: ChromaClient times out on Vercel's Edge network

[DaveCBeck]

What happened?

const client = new ChromaClient({
    path: process.env.CHROMA_URL,
  });
console.log(client.heartbeat())

Is timing out on Vercel's Edge network, without sending a request through to the server that's hosting Chroma. I can access the server fine from other projects, or with the same file but not using the edge runtime. But not on the Edge.

There's no error returned, it just hangs.

I'm guessing its something to do with the APIs supported on the Edge by Vercel: https://vercel.com/docs/functions/edge-functions/edge-runtime#supported-apis ??

@moreno1123 reported a similar problem on langchain, but I think it's a chromaclient issue, hence raising here. https://github.com/langchain-ai/langchainjs/issues/3521

Versions

ChromaDB 1.6.1

[HammadB]

That is interesting, do you have a stack trace you could share? Not sure what the env differences on vercel edge functions could be.

[jeffchuber]

I also raised this with the Vercel team

On Tue, Dec 12, 2023 at 9:01 AM Hammad Bashir @.***> wrote:

That is interesting, do you have a stack trace you could share? Not sure what the env differences on vercel edge functions could be.

— Reply to this email directly, view it on GitHub https://github.com/chroma-core/chroma/issues/1502#issuecomment-1852443057, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGZWEGEJCIMB4K7HHB7UPDYJCEV5AVCNFSM6AAAAABARQNTHWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSGQ2DGMBVG4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[DaveCBeck]

Thanks both.

@HammadB I attempted to get a stack trace using this code, but nothing particularly useful in the logs.

// Connecting to Chroma with a timeout to try and catch the stack
  const timeout = (ms: any) => new Promise((_, reject) => setTimeout(() => reject(new Error('Operation timed out')), ms));

  try {
    const client = new ChromaClient({
      path: process.env.CHROMA_URL,
    });
    console.trace();
    // Use Promise.race to throw an error if the heartbeat takes longer than 5 seconds
    const result = await Promise.race([
      client.heartbeat(),
      timeout(5000) // 5 seconds timeout
    ]);
    console.log(result);
  } catch (error) {
    const e = error as Error;
    console.error('Error occurred:', e.message);
    // Couple of different attempts at getting a stack trace
    console.error('Stack Trace:', e.stack);
    console.trace();
  }

Logs

Error occurred: Operation timed out
Stack Trace: Error: Operation timed out
    at (app/api/chat/route.ts:57:82)

The route.ts in question is here for context, but its only very slightly adapted from a Vercel template: https://github.com/DaveCBeck/Chat-with-LLM/blob/VercelEdge/frontend/app/api/chat/route.ts

[moreno1123]

any updates?

[DaveCBeck]

Just for info @moreno1123, as a temporary workaround (assuming Vercel or Chroma will sort this at some point) you can comment out the edge portion of the code and it runs fine. Obviously not for production use but for testing or low-volume use I've not had any problems.

app/api/chat/route.ts

// off the Edge for now, because otherwise the ChromaClient times out without sending a request to the server.
//export const runtime = 'edge'
export const maxDuration = 300; // comment this out if running on the edge

[moreno1123]

@DaveCBeck thanks. I did that but it was not useful in my case. I ended up adding one request between and it was somewhat ok, since it's a demo educational app and most requests are simple. Ultimately I'm migrating to llama-index... I find it more reliable and less complex than langchain.