`notIn` Prisma operator missing in `PrismaSqlFilter`

standuprey commented 4 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain.js documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain.js rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The following code:

export function genRemovedIdsFilter(
  removedIds: number[]
): PrismaSqlFilter<CompanyDetails> | null {
  if (!removedIds.length) return null
  return {
    companyId: {
      notIn: removedIds,
    },
  }
}

Error Message and Stack Trace (if applicable)

No response

Description

An error is saying 'notIn' does not exist in type '{ equals?: number; in?: number[]; isNull?: number; isNotNull?: number; like?: number; lt?: number; lte?: number; gt?: number; gte?: number; not?: number; }'

unsurprisingly, the notIn Prisma operator missing in PrismaSqlFilter:

export type PrismaSqlFilter<TModel extends Record<string, unknown>> = {
    [K in keyof TModel]?: {
        equals?: TModel[K];
        in?: TModel[K][];
        isNull?: TModel[K];
        isNotNull?: TModel[K];
        like?: TModel[K];
        lt?: TModel[K];
        lte?: TModel[K];
        gt?: TModel[K];
        gte?: TModel[K];
        not?: TModel[K];
    };
};

This issue was already mentioned here: https://github.com/langchain-ai/langchainjs/issues/692#issuecomment-1627462362

System Info

└─ langchain@npm:0.1.36
   ├─ Instances: 1
   ├─ Version: 0.1.36
   │
   └─ Dependencies
      ├─ @anthropic-ai/sdk@npm:^0.9.1 → npm:0.9.1
      ├─ @langchain/community@npm:~0.0.47 → npm:0.0.55
      ├─ @langchain/core@npm:~0.1.60 → npm:0.1.62
      ├─ @langchain/openai@npm:~0.0.28 → npm:0.0.28
      ├─ @langchain/textsplitters@npm:~0.0.0 → npm:0.0.0
      ├─ binary-extensions@npm:^2.2.0 → npm:2.3.0
      ├─ js-tiktoken@npm:^1.0.7 → npm:1.0.10
      ├─ js-yaml@npm:^4.1.0 → npm:4.1.0
      ├─ jsonpointer@npm:^5.0.1 → npm:5.0.1
      ├─ langchainhub@npm:~0.0.8 → npm:0.0.8
      ├─ langsmith@npm:~0.1.7 → npm:0.1.13
      ├─ ml-distance@npm:^4.0.0 → npm:4.0.1
      ├─ openapi-types@npm:^12.1.3 → npm:12.1.3
      ├─ p-retry@npm:4 → npm:4.6.2
      ├─ uuid@npm:^9.0.0 → npm:9.0.1
      ├─ yaml@npm:^2.2.1 → npm:2.4.1
      ├─ zod-to-json-schema@npm:^3.22.3 → npm:3.22.4
      └─ zod@npm:^3.22.4 → npm:3.22.4

dosubot[bot] commented 4 months ago

It seems the notIn operator is indeed missing from the PrismaSqlFilter type definition in LangChain.js, which aligns with the error you're encountering. This omission means that directly using notIn within PrismaSqlFilter isn't supported in the current implementation.

For a workaround, consider using the addNotClause method as an alternative approach to achieve a similar effect. This method allows for negating conditions, which could potentially mimic the notIn behavior by creatively using negation with other operations. Here's how you might use it:

addNotClause(column: string, operator: string, value: unknown) {
  this.values.push([column, `not.${operator}.${value}`]);
}

This snippet demonstrates applying a not operation, which can simulate notIn by negating an in condition or similar. Keep in mind, this is a workaround and might not cover all use cases as directly as notIn would.

#### Details

**Sources** - [libs/langchain-community/src/vectorstores/prisma.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-community/src/vectorstores/prisma.ts) - [langchain/src/retrievers/self_query/supabase_utils.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/retrievers/self_query/supabase_utils.ts)
**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

standuprey commented 4 months ago

@dosu I am not seeing any addNotClause in the documentation or source code, can point it out to me? which package?

dosubot[bot] commented 4 months ago

It appears there was a misunderstanding regarding the addNotClause method, as it's not directly related to Prisma or the PrismaSqlFilter type definition but rather to handling queries within LangChain.js, specifically for Supabase utilities. The method I mentioned doesn't exist in the context you're working with and was an error in my explanation.

For implementing a notIn functionality with Prisma in LangChain.js, since the notIn operator is missing from the PrismaSqlFilter, you would typically look for alternative approaches within Prisma's querying capabilities. However, since the context has shifted towards Supabase and not Prisma, my previous advice was misplaced.

If you're working with Supabase in LangChain.js and need to apply a notIn logic, you would directly use Supabase's querying functions, which allow for a wide range of conditional operations, including negations and exclusions, without needing a workaround like an addNotClause method.

For Prisma, if you need to implement a notIn condition and it's not supported in your current version or setup, you might consider updating your Prisma version or checking Prisma's documentation for the latest on how to handle such cases, as Prisma regularly updates its query capabilities and type definitions.

I apologize for the confusion and hope this clarifies the situation.

#### Details

**Sources** - [libs/langchain-community/src/vectorstores/prisma.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-community/src/vectorstores/prisma.ts) - [langchain/src/retrievers/self_query/supabase_utils.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/retrievers/self_query/supabase_utils.ts)
**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

jeasonnow commented 4 months ago

It seems that the logic for the notIn operator has not been implemented yet, but it appears to be highly similar to the in operator. If this approach works, I can submit a pull request. @standuprey

https://github.com/jeasonnow/langchainjs/blob/1f2f1cf56e33b3c3a6136bea82e9920aac31ad8c/libs/langchain-community/src/vectorstores/prisma.ts#L416-L457

dosubot[bot] commented 4 months ago

Implementing the notIn logic in the PrismaSqlFilter type definition for LangChain.js, similarly to the in operator, is indeed a viable solution for the missing notIn operator issue. You can proceed with submitting a pull request. Here's a concise guide on how to add notIn functionality:

Extend PrismaSqlFilter Type: Add notIn to the type definition to include arrays of the model's type.
```
notIn?: TModel[K][];
```
Update the OpMap: Include notIn in the operation mapping with the SQL NOT IN operation.
```
notIn: "NOT IN",
```
Implement notIn Logic: Modify the buildSqlFilterStr method to handle notIn by constructing the appropriate SQL query part.
```
case OpMap.notIn: {
 // Ensure value is an array and construct the SQL part
}
```

Remember to update or extend the existing tests to cover the new notIn operator, ensuring it behaves as expected within the framework.

When implementing notIn, consider the potential performance implications, especially with large exclusion lists, as it might affect query efficiency depending on the database's indexing and configuration.

Ensure your contribution adheres to the LangChain JS repository's coding standards and guidelines, including following the "fork and pull request" workflow, adhering to linting rules, including tests, and documenting your changes. Check the CONTRIBUTING.md document and the integration contribution guide for more details on contributing to LangChain JS.

#### Details

**Sources** - [libs/langchain-community/src/vectorstores/prisma.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-community/src/vectorstores/prisma.ts) - [libs/langchain-community/src/vectorstores/tests/prisma.test.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-community/src/vectorstores/tests/prisma.test.ts) - [.github/contributing/INTEGRATIONS.md](https://github.com/langchain-ai/langchainjs/blob/main/.github/contributing/INTEGRATIONS.md) - [CONTRIBUTING.md](https://github.com/langchain-ai/langchainjs/blob/main/CONTRIBUTING.md)
**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

standuprey commented 4 months ago

Yes, that'd be a great improvement as far as I'm concerned

langchain-ai / langchainjs