langchain-airbyte broken dependencies

[cbornet]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

cd libs/partners/airbyte
poetry lock --no-update

you get

Because langchain (0.3.1) @ file:///Users/christophebornet/workspace/langchain/libs/langchain depends on pydantic (^2.7.4)
 and langchain-airbyte depends on pydantic (>=1.10.8,<2), langchain is forbidden.
So, because langchain-airbyte depends on langchain (0.3.1) @ file:///Users/christophebornet/workspace/langchain/libs/langchain, version solving failed.

Description

• langchain-airbyte dependencies are broken because pydantic is forced to v1 and langchain 0.3+ uses pydantic v2
• newer versions of langchain-airbyte work with pydantic v2 but airbyte depends on airbyte-cdk which depends on the pinned version langchain-core 0.1.42! So there is a circular dependency with no compatible combination at the moment... If we remove the pydantic constraint and update airbyte to a version that supports pydantic v2, we get:

  Because airbyte-cdk (2.3.1) depends on langchain_core (0.1.42)
  and airbyte-cdk (2.3.0) depends on langchain_core (0.1.42), airbyte-cdk (2.3.0 || 2.3.1) requires langchain_core (0.1.42).
  And because airbyte-cdk (2.2.0) depends on langchain_core (0.1.42)
  and airbyte-cdk (2.1.0) depends on langchain_core (0.1.42), airbyte-cdk (2.1.0 || 2.2.0 || 2.3.0 || 2.3.1) requires 
  langchain_core (0.1.42).
  And because airbyte-cdk (2.0.0) depends on langchain_core (0.1.42)
  and no versions of airbyte-cdk match >2.0,<2.1.0 || >2.1.0,<2.2.0 || >2.2.0,<2.3.0 || >2.3.0,<2.3.1 || >2.3.1,<2.4.0 || >2.4.0, 
  <3.0, airbyte-cdk (>=2.0,<2.4.0 || >2.4.0,<3.0) requires langchain_core (0.1.42).
  And because airbyte-cdk (2.4.0) depends on langchain_core (0.1.42)
  and airbyte-cdk (3.0.0) depends on langchain_core (0.1.42), airbyte-cdk (>=2.0,<=3.0) requires langchain_core (0.1.42).
  Because no versions of airbyte match >0.13.0,<0.14.0
  and airbyte (0.13.0) depends on airbyte-cdk (>=2.0,<=3.0), airbyte (>=0.13.0,<0.14.0) requires airbyte-cdk (>=2.0, 
  <=3.0).
  Thus, airbyte (>=0.13.0,<0.14.0) requires langchain_core (0.1.42).
  So, because langchain-airbyte depends on both langchain-core (0.3.6) @ 
  file:///Users/christophebornet/workspace/langchain/libs/core and airbyte (^0.13.0), version solving failed.

System Info

System Information
------------------
> OS:  Darwin
> OS Version:  Darwin Kernel Version 22.6.0: Mon Feb 19 19:48:53 PST 2024; root:xnu-8796.141.3.704.6~1/RELEASE_X86_64
> Python Version:  3.11.9 (main, Jul 25 2024, 22:35:19) [Clang 15.0.0 (clang-1500.0.40.1)]

Package Information
-------------------
> langchain_core: Installed. No version info available.

Optional packages not installed
-------------------------------
> langsmith
> langgraph
> langserve

[keenborder786]

This is fixable but first we need to create a PR here, which updates the airbyte-cdk to support newer version of langchain_core. And then we can use the following dependencies. This should solve the issue.

[tool.poetry.dependencies]
python = ">=3.9,<3.12.4"
langchain-core = "^0.3"
airbyte = "^0.11.0"
airbyte-cdk = "^3.0.0"