TypeError: Converting circular structure to JSON when enabling LangSmith

sinedied commented 6 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain.js documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain.js rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The issue occurs when enabling Langsmith on this sample, in particular the Azure path: https://github.com/Azure-Samples/serverless-chat-langchainjs

The issue seems to come from the AzureCosmosDBVectorStore. I'm trying to investigate what could be the cause of the circular structure, do you have any hint on how to debug which object is the root cause?

Error Message and Stack Trace (if applicable)

[api] [2024-04-23T15:11:38.138Z] TypeError: Converting circular structure to JSON
[api] [2024-04-23T15:11:38.138Z]     --> starting at object with constructor 'Socket'
[api] [2024-04-23T15:11:38.138Z]     |     property 'parser' -> object with constructor 'HTTPParser'
[api] [2024-04-23T15:11:38.138Z]     --- property 'socket' closes the circle
[api] [2024-04-23T15:11:38.138Z]     at JSON.stringify (<anonymous>)
[api] [2024-04-23T15:11:38.138Z]     at Client.batchIngestRuns (/Users/sinedied/projects/serverless-chat-langchainjs/node_modules/langsmith/dist/client.cjs:637:51)
[api] [2024-04-23T15:11:38.138Z]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[api] [2024-04-23T15:11:38.139Z]     at async Client.drainAutoBatchQueue (/Users/sinedied/projects/serverless-chat-langchainjs/node_modules/langsmith/dist/client.cjs:477:17)

Description

Trying to enable Langsmith, but the app crashes when trying to send the report.

System Info

npm ls langchain @langchain/comm unity serverless-chat-langchainjs@1.0.0 /Users/sinedied/projects/serverless-chat-langchainjs └─┬ api@1.0.0 -> ./packages/api ├── @langchain/community@0.0.51 -> ./packages/api └─┬ langchain@0.1.35 └── @langchain/community@0.0.51 deduped -> ./packages/api

platform mac, node v20.11.0, npm 10.2.4

dosubot[bot] commented 6 months ago

To address the "TypeError: Converting circular structure to JSON" issue you're encountering with LangSmith, particularly in the context of AzureCosmosDBVectorStore, the problem likely stems from a circular reference within the objects being serialized. Here's a focused approach to tackle this:

Identify and Debug Circular References: Utilize console.log() or Node.js's util.inspect() to inspect objects just before serialization. This can help pinpoint where the circular reference occurs.

Custom Replacer for JSON.stringify(): Implement a custom replacer function to handle circular references during JSON serialization. Here's a concise example:

function getCircularReplacer() {
 const seen = new WeakSet();
 return (key, value) => {
   if (typeof value === "object" && value !== null) {
     if (seen.has(value)) {
       return;
     }
     seen.add(value);
   }
   return value;
 };
}
JSON.stringify(obj, getCircularReplacer());

Apply and Test: After applying the above solution, thoroughly test to ensure the issue is resolved without side effects.

The error trace you provided points to Client.batchIngestRuns, suggesting the issue might not be directly within AzureCosmosDBVectorStore but possibly in the objects it handles or the method itself. Apply the debugging steps to this area as well.

If the problem persists, providing more details about the operations leading to the error could help in offering a more specific solution.

#### Details

**Sources** - [libs/langchain-community/src/vectorstores/azure_cosmosdb.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-community/src/vectorstores/azure_cosmosdb.ts) - [langchain-core/src/utils/@cfworker/json-schema/src/dereference.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain-core/src/utils/@cfworker/json-schema/src/dereference.ts) - [examples/src/indexes/vector_stores/azure_cosmosdb/azure_cosmosdb.ts](https://github.com/langchain-ai/langchainjs/blob/main/examples/src/indexes/vector_stores/azure_cosmosdb/azure_cosmosdb.ts) - [langchain-core/src/tracers/console.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain-core/src/tracers/console.ts) - [langchain-core/src/callbacks/manager.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain-core/src/callbacks/manager.ts) - [langchain/src/smith/runner_utils.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/smith/runner_utils.ts)
**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

sinedied commented 6 months ago

Ok, removing entirely AzureCosmosDBVectorStore from the chain did not solve the issue. I still get that error, but it cannot be reproduced all the time. Not sure how I should go to debug this or help debugging this?

Kniggishood commented 5 months ago

Heyhey, can you provide the code sample for this? What does your wrappedFunc look like?

sinedied commented 5 months ago

Yes, I could reproduce the error on this sample/branch: https://github.com/Azure-Samples/serverless-chat-langchainjs/tree/mongodb-vcore

I'm currently off till next week, I'll try to create a simpler repo for reproducing the issue.

pnedelko commented 3 months ago

Faced the same issue when trying to implement my custom cache Class.

TLDR: with Langsmith tracing I get error, without tracings works well.

The error:

Error in handler LangChainTracer, handleChainEnd: TypeError: Converting circular structure to JSON
     --> starting at object with constructor 'ni'
     |     property 'client' -> object with constructor 'PrismaService'
     --- property '_requestHandler' closes the circle

The problem appears when there is a class prop with PrismaClient

class PrismaCache extends BaseCache {
   constructor(
     private prisma: PrismaClient,
     private options: { ttl: number },
   ) {
     super();
   }
...
}

Temporal workaround: pass an object to the constructor that holds a reference to the PrismaClient in the closure.

{
      get: async (key: string) => {
        const entry = await this.prisma.llmCache.findUnique({
          where: {
            key,
            expiresAt: {
              gt: new Date(),
            },
          },
        });

        if (!entry) {
          return null;
        }

        return entry.value;
      },
      set: async (key: string, value: any, ttl: number) => {
        await this.prisma.llmCache.upsert({
          where: {
            key,
          },
          create: {
            key,
            value: value as any,
            expiresAt: new Date(Date.now() + ttl),
          },
          update: {
            value: value as any,
            expiresAt: new Date(Date.now() + ttl),
          },
        });
      },
    }

airhorns commented 3 months ago

I'm experiencing this issue as well -- it seems like Langsmith tries to blindly serialize the arguments and return values of traced functions, and if those values happen to be rich objects that reference one another, they can't be safely serialized like that. Could we get an API added to allow a transform of the values before being serialized? Some of the objects that my functions use are also quite big if serialized -- maybe it's the wrong default to serialize everything blindly?

airhorns commented 3 months ago

Another option would be to use one of the circular-safe serialization libraries

mbennett-talentnet commented 2 months ago

Any update on this? I have enabled MongoDBSaver for checkpointing in langgraph and now langsmith is broken with the following error:

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'MongoClient'
    |     property 's' -> object with constructor 'Object'
    |     property 'sessionPool' -> object with constructor 'ServerSessionPool'
    --- property 'client' closes the circle
    at JSON.stringify (<anonymous>)
    at Client.batchIngestRuns (file:///var/www/app/node_modules/langsmith/dist/client.js:612:51)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Client.drainAutoBatchQueue (file:///var/www/app/node_modules/langsmith/dist/client.js:451:17)

dosubot[bot] commented 2 months ago

To debug the "TypeError: Converting circular structure to JSON" error when enabling Langsmith in the AzureCosmosDBVectorStore of the serverless-chat-langchainjs application, you need to identify where the circular structure is being created. This error typically occurs when trying to stringify an object that has circular references.

Here are some steps to help you debug this issue:

Check Object Creation: Ensure that the objects being created do not have circular references. For example, in the addVectors method, check the docs array.
Use a Circular Reference Detector: You can use a library like circular-json to help detect and handle circular references.
Add Debugging Statements: Add console logs to inspect the objects before they are stringified or sent to the database.

Here is an example of how you can modify the addVectors method to include debugging statements and use circular-json:

import CircularJSON from 'circular-json';

async addVectors(
  vectors: number[][],
  documents: DocumentInterface[]
): Promise<string[]> {
  await this.initialize();
  const docs = vectors.map((embedding, idx) => ({
    [this.textKey]: documents[idx].pageContent,
    [this.embeddingKey]: embedding,
    [this.metadataKey]: documents[idx].metadata,
    ...(documents[idx].id ? { id: documents[idx].id } : {}),
  }));

  // Debugging: Check for circular references
  try {
    CircularJSON.stringify(docs);
  } catch (error) {
    console.error("Circular reference detected in docs:", error);
    throw error;
  }

  const ids: string[] = [];
  const results = await Promise.all(
    docs.map((doc) => this.container.items.create(doc))
  );

  for (const result of results) {
    ids.push(result.resource?.id ?? "error: could not create item");
  }

  return ids;
}

By adding the CircularJSON.stringify(docs) check, you can catch circular references early and log them for further inspection. This should help you identify the source of the circular structure and resolve the error ^[1].

To continue talking to Dosu, mention @dosu.

jacoblee93 commented 2 months ago

Thanks for reporting - will identify and patch as soon as possible!

jacoblee93 commented 2 months ago

PR opened here: https://github.com/langchain-ai/langsmith-sdk/pull/985

langchain-ai / langchainjs