search

langchain-ai / langchainjs

🦜🔗 Build context-aware reasoning applications 🦜🔗
https://js.langchain.com/docs/
MIT License
12.29k stars 2.08k forks source link

withStructuredOutput is not a function on ChatGoogleGenerativeAI #5484

Closed yharaskrik closed 3 months ago

yharaskrik commented 3 months ago

Checked other resources

Example Code

const genAi = new ChatGoogleGenerativeAI({});

console.log(genAi.withStructuredOutput); <-- yields undefined, calling it on the call results in below

Error Message and Stack Trace (if applicable)

}).withStructuredOutput(schema);
   ^

TypeError: (intermediate value).withStructuredOutput is not a function

Description

The ChatGoogleGenerativeAI class has the withStructuredOutput function on it, but it is undefined. If it is not implemented for this model then it should not be on the types.

        "@langchain/community": "^0.2.0",
        "@langchain/core": "^0.2.0",
        "@langchain/google-genai": "^0.0.14",
        "@langchain/openai": "^0.0.33",
        "@langchain/redis": "^0.0.5",
        "langchain": "^0.2.0",

System Info

Mac yarn 1.22.7 node 18.20.2

dosubot[bot] commented 3 months ago

Hey there, @yharaskrik! 👋 I'm Dosu, a bot here to lend a hand with bugs, answer questions, and guide you through becoming a contributor while we wait for a human maintainer to join us. Think of me as your friendly digital sidekick in navigating the world of LangChain JS. I'm diving into your issue now and will get back to you with a full answer shortly. Hang tight! 🤖

yharaskrik commented 3 months ago

Oh I see the withStructuredOutput function is optional on the abstract BaseLanguageModel class. My IDE was not reflecting that, so the ChatGoogleGenerativeAI is not implementing it and that is why it is undefined.

bracesproul commented 3 months ago

The withStructuredOutput isn't implemented (or normal tool calling for that matter) in the @langchain/google-genai package. It's on my list to implement both sometime this week though! I'll use this issue as a tracker for when that goes in.

yharaskrik commented 3 months ago

Amazing thank you Brace! That's pretty much what I determined as well. I went to try and do it myself but only got so far as copying the code from the common package for those functions. I'm sure there's a better way just not familiar with the structure of all the packages!

Thanks!

On Mon, May 20, 2024, 11:11 AM Brace Sproul @.***> wrote:

The withStructuredOutput isn't implemented (or normal tool calling for that matter) in the @langchain/google-genai package. It's on my list to implement both sometime this week though! I'll use this issue as a tracker for when that goes in.

— Reply to this email directly, view it on GitHub https://github.com/langchain-ai/langchainjs/issues/5484#issuecomment-2120954193, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACIHZITXNRW2OJXSVPZHXNTZDI4GNAVCNFSM6AAAAABIAD455GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRQHE2TIMJZGM . You are receiving this because you were mentioned.Message ID: @.***>

bracesproul commented 3 months ago

g

That's how I was planning on going about it as well! If you want to take a stab at it, I wouldn't start until tmro. So if you can put up a PR before tmro morning I would love to use your implementation!

dosubot[bot] commented 3 months ago

To implement the withStructuredOutput function in the @langchain/google-genai package, follow these steps:

  1. Fork and Clone the Repository: Fork the langchainjs repository to your account and clone it locally.

  2. Set Up Your Development Environment: Ensure Node.js (v18+) is installed. Run yarn install in the repository's root to install dependencies.

  3. Create a New Branch: Use git checkout -b feature/withStructuredOutput to create a branch for your changes.

  4. Implement withStructuredOutput:

    • Navigate to langchain-core/src/language_models.
    • Update or define the withStructuredOutput function in the relevant TypeScript files. Follow the project's coding standards and ensure compatibility with existing patterns.

  5. Build and Test Your Changes: Run yarn build to build the package and yarn test to ensure your changes don't break existing functionality. Add new tests if necessary.

  6. Commit and Push Your Changes: After testing, commit your changes and push them to your fork.

  7. Open a Pull Request (PR): From your fork, open a PR to the main langchainjs repository. Describe your changes and their motivation clearly.

  8. Review Process: Engage with any review feedback to refine your contribution.

This approach will help you contribute effectively to resolving the TypeError issue with withStructuredOutput in the @langchain/google-genai package.

#### Details **Sources** - [OpenAI Agent returning structured output has type errors](https://github.com/langchain-ai/langchainjs/issues/3774) - [libs/langchain-google-genai/package.json](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-google-genai/package.json) - [libs/langchain-google-common/src/chat_models.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-google-common/src/chat_models.ts) - [langchain-core/src/language_models/base.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain-core/src/language_models/base.ts) - [libs/langchain-openai/src/chat_models.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-openai/src/chat_models.ts) - [langchain/src/chains/openai_functions/structured_output.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/chains/openai_functions/structured_output.ts) - [langchain/src/chains/openai_functions/tests/structured_output.test.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/chains/openai_functions/tests/structured_output.test.ts) - [langchain/src/chains/openai_functions/base.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/chains/openai_functions/base.ts) - [langchain/src/chains/openai_functions/tests/structured_output.int.test.ts](https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/chains/openai_functions/tests/structured_output.int.test.ts) - [docs/core_docs/docs/how_to/structured_output.ipynb](https://github.com/langchain-ai/langchainjs/blob/main/docs/core_docs/docs/how_to/structured_output.ipynb) - [libs/langchain-openai/src/tests/chat_models_structured_output.int.test.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-openai/src/tests/chat_models_structured_output.int.test.ts) - [libs/langchain-groq/src/tests/chat_models_structured_output.int.test.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-groq/src/tests/chat_models_structured_output.int.test.ts) - [libs/langchain-anthropic/src/tests/chat_models.test.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-anthropic/src/tests/chat_models.test.ts) - [libs/langchain-google-genai/src/chat_models.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-google-genai/src/chat_models.ts) - [examples/src/models/chat/googlegenerativeai.ts](https://github.com/langchain-ai/langchainjs/blob/main/examples/src/models/chat/googlegenerativeai.ts) - [libs/langchain-anthropic/src/experimental/tool_calling.ts](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-anthropic/src/experimental/tool_calling.ts) - [libs/langchain-google-genai/README.md](https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-google-genai/README.md)
**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

bitsnaps commented 3 months ago

The same applies for huggingface inference API, I'm getting the error TypeError: llm.withStructuredOutput is not a function even with models supporting function calling.

jason-allen-oneal commented 2 months ago

same with groq