Luc2357 commented 4 days ago

Upon creation of a new virtual environment, the import of the ChatVertexAI now fails with "'SafetySetting' is not defined"

Steps to reproduce:

python3 -m venv .venv
source .venv/bin/activate
pip install langchain-google-vertexai
python -c "from langchain_google_vertexai import ChatVertexAI"

Error reported:

NameError: name 'SafetySetting' is not defined. Did you mean: 'SafetySettingsType'?

Full stacktrace

``` Traceback (most recent call last): File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 815, in _resolve_forward_ref obj = _typing_extra.eval_type_backport(obj, *self._types_namespace) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_typing_extra.py", line 534, in eval_type_backport return _eval_type_backport(value, globalns, localns, type_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_typing_extra.py", line 558, in _eval_type_backport return _eval_type(value, globalns, localns, type_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_typing_extra.py", line 592, in _eval_type return typing._eval_type( # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 414, in _eval_type return t._evaluate(globalns, localns, recursive_guard) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 929, in _evaluate self.__forward_value__ = _eval_type( ^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 428, in _eval_type ev_args = tuple(_eval_type(a, globalns, localns, recursive_guard) for a in t.__args__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 428, in ev_args = tuple(_eval_type(a, globalns, localns, recursive_guard) for a in t.__args__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 414, in _eval_type return t._evaluate(globalns, localns, recursive_guard) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 929, in _evaluate self.__forward_value__ = _eval_type( ^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 428, in _eval_type ev_args = tuple(_eval_type(a, globalns, localns, recursive_guard) for a in t.__args__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 428, in ev_args = tuple(_eval_type(a, globalns, localns, recursive_guard) for a in t.__args__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 428, in _eval_type ev_args = tuple(_eval_type(a, globalns, localns, recursive_guard) for a in t.__args__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 428, in ev_args = tuple(_eval_type(a, globalns, localns, recursive_guard) for a in t.__args__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 414, in _eval_type return t._evaluate(globalns, localns, recursive_guard) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 924, in _evaluate eval(self.__forward_code__, globalns, localns), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1, in NameError: name 'SafetySetting' is not defined. Did you mean: 'SafetySettingsType'? The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 1, in File "/Users/lucpons/.venv/lib/python3.12/site-packages/langchain_google_vertexai/__init__.py", line 16, in from langchain_google_vertexai.embeddings import VertexAIEmbeddings File "/Users/lucpons/.venv/lib/python3.12/site-packages/langchain_google_vertexai/embeddings.py", line 544, in VertexAIEmbeddings.model_rebuild() File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/main.py", line 589, in model_rebuild return _model_construction.complete_model_class( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_model_construction.py", line 658, in complete_model_class schema = cls.__get_pydantic_core_schema__(cls, handler) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/main.py", line 697, in __get_pydantic_core_schema__ return handler(source) ^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__ schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 612, in generate_schema schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 881, in _generate_schema_inner return self._model_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 693, in _model_schema {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()}, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1073, in _generate_md_field_schema common_field = self._common_field_schema(name, field_info, decorators) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1261, in _common_field_schema schema = self._apply_annotations( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2051, in _apply_annotations schema = get_inner_schema(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__ schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2032, in inner_handler schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 875, in _generate_schema_inner return self.generate_schema(self._resolve_forward_ref(obj)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/lucpons/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 817, in _resolve_forward_ref raise PydanticUndefinedAnnotation.from_name_error(e) from e pydantic.errors.PydanticUndefinedAnnotation: name 'SafetySetting' is not defined For further information visit https://errors.pydantic.dev/2.10/u/undefined-annotation ```

fazpu commented 4 days ago

same here. Here is my stack trace when running unit tests on CI:

llms/fallback_llm_provider.py:14: in <module>
    from langchain_google_vertexai import ChatVertexAI
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/langchain_google_vertexai/__init__.py:16: in <module>
    from langchain_google_vertexai.embeddings import VertexAIEmbeddings
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/langchain_google_vertexai/embeddings.py:544: in <module>
    VertexAIEmbeddings.model_rebuild()
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/main.py:589: in model_rebuild
    return _model_construction.complete_model_class(
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_model_construction.py:658: in complete_model_class
    schema = cls.__get_pydantic_core_schema__(cls, handler)
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/main.py:[69](https://github.com/writeitai/writeit/actions/runs/11953818231/job/33323404662#step:10:70)7: in __get_pydantic_core_schema__
    return handler(source)
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py:84: in __call__
    schema = self._handler(source_type)
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:612: in generate_schema
    schema = self._generate_schema_inner(obj)
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:881: in _generate_schema_inner
    return self._model_schema(obj)
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:693: in _model_schema
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:693: in <dictcomp>
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:10[73](https://github.com/writeitai/writeit/actions/runs/11953818231/job/33323404662#step:10:74): in _generate_md_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:1261: in _common_field_schema
    schema = self._apply_annotations(
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:2051: in _apply_annotations
    schema = get_inner_schema(source_type)
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py:84: in __call__
    schema = self._handler(source_type)
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:2032: in inner_handler
    schema = self._generate_schema_inner(obj)
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:8[75](https://github.com/writeitai/writeit/actions/runs/11953818231/job/33323404662#step:10:76): in _generate_schema_inner
    return self.generate_schema(self._resolve_forward_ref(obj))
/home/runner/.local/share/hatch/env/virtual/code-bro-server/v7rDU5c8/test/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:817: in _resolve_forward_ref
    raise PydanticUndefinedAnnotation.from_name_error(e) from e
E   pydantic.errors.PydanticUndefinedAnnotation: name 'SafetySetting' is not defined

mkesicki commented 4 days ago

Our docker:


boto3==1.34.22
google-api-python-client==2.114.0
google-auth==2.26.2
langchain==0.3.2
langchain-core==0.3.9
langchain-google-vertexai==2.0.3
langgraph==0.2.18
pdf2image==1.17.0
pillow==10.2.0```

Logs:
```For further information visit https://errors.pydantic.dev/2.10/u/undefined-annotation
Traceback (most recent call last):
  File "/var/lang/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/var/task/datadog_lambda/handler.py", line 30, in <module>
    handler_module = import_module(modified_mod_name)
  File "/var/lang/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/var/task/ddtrace/internal/module.py", line 295, in _exec_module
    self.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/var/task/lambda.py", line 3, in <module>
    import main
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/var/task/ddtrace/internal/module.py", line 295, in _exec_module
    self.loader.exec_module(module)
  File "/var/task/main.py", line 8, in <module>
    from langchain_google_vertexai import HarmBlockThreshold, HarmCategory, ChatVertexAI
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/var/task/ddtrace/internal/module.py", line 295, in _exec_module
    self.loader.exec_module(module)
  File "/var/task/langchain_google_vertexai/__init__.py", line 16, in <module>
    from langchain_google_vertexai.embeddings import VertexAIEmbeddings
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/var/task/ddtrace/internal/module.py", line 295, in _exec_module
    self.loader.exec_module(module)
  File "/var/task/langchain_google_vertexai/embeddings.py", line 544, in <module>
    VertexAIEmbeddings.model_rebuild()
  File "/var/task/pydantic/main.py", line 589, in model_rebuild
    return _model_construction.complete_model_class(
  File "/var/task/pydantic/_internal/_model_construction.py", line 658, in complete_model_class
    schema = cls.__get_pydantic_core_schema__(cls, handler)
  File "/var/task/pydantic/main.py", line 697, in __get_pydantic_core_schema__
    return handler(source)
  File "/var/task/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__
    schema = self._handler(source_type)
  File "/var/task/pydantic/_internal/_generate_schema.py", line 612, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/var/task/pydantic/_internal/_generate_schema.py", line 881, in _generate_schema_inner
    return self._model_schema(obj)
  File "/var/task/pydantic/_internal/_generate_schema.py", line 693, in _model_schema
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
  File "/var/task/pydantic/_internal/_generate_schema.py", line 693, in <dictcomp>
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
  File "/var/task/pydantic/_internal/_generate_schema.py", line 1073, in _generate_md_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
  File "/var/task/pydantic/_internal/_generate_schema.py", line 1261, in _common_field_schema
    schema = self._apply_annotations(
  File "/var/task/pydantic/_internal/_generate_schema.py", line 2051, in _apply_annotations
    schema = get_inner_schema(source_type)
  File "/var/task/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__
    schema = self._handler(source_type)
  File "/var/task/pydantic/_internal/_generate_schema.py", line 2032, in inner_handler
    schema = self._generate_schema_inner(obj)
  File "/var/task/pydantic/_internal/_generate_schema.py", line 875, in _generate_schema_inner
    return self.generate_schema(self._resolve_forward_ref(obj))
  File "/var/task/pydantic/_internal/_generate_schema.py", line 817, in _resolve_forward_ref
    raise PydanticUndefinedAnnotation.from_name_error(e) from e

jzaldi commented 4 days ago

As a workaround setting pydantic version to 2.9.0 seems to work. 2.10.0 seems to break backwards compatibility somehow. We have pydantic version bounded between 2 and 3 so new install might be pulling the new version. This works for me

virtualenv .venv
source .venv/bin/activate
pip install langchain-google-vertexai
pip install pydantic==2.9.1
python -c "from langchain_google_vertexai import ChatVertexAI"

tiaan720 commented 4 days ago

We also received the same error. Weirdly enough not on our local tests but in the CI/CD pipelines tests. I tried this:

virtualenv .venv
source .venv/bin/activate
pip install langchain-google-vertexai
pip install pydantic==2.9.1
python -c "from langchain_google_vertexai import ChatVertexAI"

But It still doesn't work.

jzaldi commented 4 days ago

We also received the same error. Weirdly enough not on our local tests but in the CI/CD pipelines tests. I tried this:
virtualenv .venv
source .venv/bin/activate
pip install langchain-google-vertexai
pip install pydantic==2.9.1
python -c "from langchain_google_vertexai import ChatVertexAI"
But It still doesn't work.

It fails with the same error?

tiaan720 commented 4 days ago

We also received the same error. Weirdly enough not on our local tests but in the CI/CD pipelines tests. I tried this:
virtualenv .venv
source .venv/bin/activate
pip install langchain-google-vertexai
pip install pydantic==2.9.1
python -c "from langchain_google_vertexai import ChatVertexAI"
But It still doesn't work.
It fails with the same error?

Same error yes E pydantic.errors.PydanticUndefinedAnnotation: name 'SafetySetting' is not defined E
E For further information visit https://errors.pydantic.dev/2.10/u/undefined-annotation

jzaldi commented 4 days ago

Can you pip freeze the .venv that is created with that command?

tiaan720 commented 4 days ago

[tool.poetry.dependencies] python = "^3.11" uvicorn = "^0.23.2" langchain-openai = "^0.2.1" langgraph = "^0.2.35" kaleido = "0.2.1" plotly = "^5.24.1" numpy = "1.26.4" python-dotenv = "^1.0.1" httpx = "^0.27.2" google-cloud-bigquery = "^3.26.0" toml = "^0.10.2" google-cloud-datastore = "^2.20.1" langchain = "^0.3.7" fastapi = "^0.115.4" pandas = "^2.2.3" langchain-google-community = "^2.0.2" langchain-google-vertexai = "^2.0.7" pydantic = "^2.9.1"

[tool.poetry.group.dev.dependencies] isort = "^5.13.2" black = "^24.10.0"

[tool.poetry.group.test.dependencies] pytest = "^8.3.3" pytest-cov = "^5.0.0" pytest-asyncio = "^0.24.0" pytest-mock = "^3.14.0" flaky = "^3.8.1"

[build-system] requires = ["poetry-core"] build-backend = "poetry.core.masonry.api"

This is our toml files dependancies

jzaldi commented 4 days ago

[tool.poetry.dependencies] python = "^3.11" uvicorn = "^0.23.2" langchain-openai = "^0.2.1" langgraph = "^0.2.35" kaleido = "0.2.1" plotly = "^5.24.1" numpy = "1.26.4" python-dotenv = "^1.0.1" httpx = "^0.27.2" google-cloud-bigquery = "^3.26.0" toml = "^0.10.2" google-cloud-datastore = "^2.20.1" langchain = "^0.3.7" fastapi = "^0.115.4" pandas = "^2.2.3" langchain-google-community = "^2.0.2" langchain-google-vertexai = "^2.0.7" pydantic = "^2.9.1"

[tool.poetry.group.dev.dependencies] isort = "^5.13.2" black = "^24.10.0"

[tool.poetry.group.test.dependencies] pytest = "^8.3.3" pytest-cov = "^5.0.0" pytest-asyncio = "^0.24.0" pytest-mock = "^3.14.0" flaky = "^3.8.1"

[build-system] requires = ["poetry-core"] build-backend = "poetry.core.masonry.api"

This is our toml files dependancies

You have pydantic ^2.9.1, this can be installing 2.10.0 where the problem seems to be. Fix this version to 2.9.2 and try again

mkesicki commented 4 days ago

FYI: The workaround works in our case both on AWS lambda & locally

Luc2357 commented 4 days ago

Thank you @jzaldi, pinning the pydantic version works!

domsj-foodpairing commented 3 days ago

I don't think this ticket should be closed already, please reopen

Of course everyone can pin their pydantic to <2.10, but some more things need to happen IMO.

temporarily restrict the pydantic version this package depends on
figure out if whether this is a regression in pydantic OR if it's a wrong usage of pydantic in langchain-google-vertexai
depending on the outcome: register pydantic bug or fix the usage here

Viicos commented 3 days ago

In Pydantic 2.10, we introduced a complete refactor of the forward annotations evaluation. As a reminder, a forward annotation is a type hint surrounded by quotes:

class Model(BaseModel):
    a: 'MyInt'

    # Or, if the `from __future__ import annotations` is included:
    b: MyInt  # the import will automatically surround the annotation with quotes

MyInt = int  # defined _after_ `Model`

Because Pydantic uses type hints to build the model schema, it needs to evaluate these annotations, which can be really challenging. Debugging issues related to forward annotations (like this one) is also complicated.

Explanation of what should happen

The 2.10 refactor revealed what seems to be a genuine bug in this library. The VertexAIEmbeddings class is a Pydantic model, using _VertexAICommon as a base class:

https://github.com/langchain-ai/langchain-google/blob/0b5d16a3d7b54c49a904de7728007084e02c2bee/libs/vertexai/langchain_google_vertexai/embeddings.py#L103-L112

_VertexAICommon has a field annotated as Optional["SafetySettingsType"] (L197):

https://github.com/langchain-ai/langchain-google/blob/0b5d16a3d7b54c49a904de7728007084e02c2bee/libs/vertexai/langchain_google_vertexai/_base.py#L176-L197

SafetySettingsType is being imported from the vertexai library. The definition can be found here:

vertexai/generative_models/_generative_models.py:

SafetySettingsType = Union[
    List["SafetySetting"],
    Dict[
        gapic_content_types.HarmCategory,
        gapic_content_types.SafetySetting.HarmBlockThreshold,
    ],
]

SafetySetting, also referenced as a forward annotation, is defined in the same file here:

class SafetySetting:
    """Parameters for the generation."""

    HarmCategory = gapic_content_types.HarmCategory
    HarmBlockMethod = gapic_content_types.SafetySetting.HarmBlockMethod
    HarmBlockThreshold = gapic_content_types.SafetySetting.HarmBlockThreshold

    ...

So if everything is working as expected, the safety_settings field annotation should resolve to something like:

class _VertexAICommon(_VertexAIBase):
    # other fields defined ...
    ...

    safety_settings: Optional[
        Union[
            List[vertexai.generative_models._generative_models.SafetySetting],
            Dict[
                gapic_content_types.HarmCategory,
                gapic_content_types.SafetySetting.HarmBlockThreshold,
            ],
        ]
    ]

What really happens in Pydantic 2.9

However, in Pydantic 2.9, 'SafetySetting' resolves to the wrong class. This can be seen by inspecting the model fields:

# pydantic==2.9.2

from langchain_google_vertexai.embeddings import VertexAIEmbeddings

VertexAIEmbeddings.model_fields['safety_settings'].annotation
>>> Optional[
>>>     Union[
>>>         List[google.cloud.aiplatform_v1beta1.types.content.SafetySetting],  # different type!
>>>         Dict[
>>>             gapic_content_types.HarmCategory,
>>>             gapic_content_types.SafetySetting.HarmBlockThreshold,
>>>         ],
>>>     ]
>>> ]

The reason we get a different type for 'SafetySetting' is quite obscure and is explained at the end of this comment.

Now two questions can be raised from this:

what is the expected type for 'SafetySetting', as referenced in SafetySettingsType? Following the Python type checking semantics, it should be vertexai.generative_models._generative_models.SafetySetting.
How can this be fixed? This isn't trivial. It seems like the vertexai library is defining a lot of aliases with forward references. This is usually hard if not impossible to support. Take the following example:
```
# module1.py
```

from typing import List

Alias = List['SomeType']

SomeType = int

module2.py

from module1 import Alias from pydantic import BaseModel

class Model(BaseModel): a: Alias

Model failed to build (calling `model_rebuild()` will raise)


Here, `a: Alias` is _strictly_ equivalent to `a: List['SomeType']`, and Pydantic has no way of knowing that `Alias` is actually defined in a separate module.

A workaround is to either avoid using forward references in such aliases (not always possible), or to use the new [`typing.TypeAliasType`](https://docs.python.org/3/library/typing.html#typing.TypeAliasType) construct:

```python
Alias = TypeAliasType('Alias', List['SomeType'])
# or, when 3.12 becomes the lowest supported version:
type Alias = List[SomeType]

Why the wrong type was used

This goes deeper in the runtime implementation of the typing constructs.

When defining such an alias:

Alias = List['SomeRef']

Any argument passed to the List constructor (or Dict, Set, etc) is implicitly converted to a ForwardRef instance.

This means that Alias is in reality List[ForwardRef('SomeRef')]. To evaluate forward annotations, we end up calling a the typing._eval_type function. This will call ForwardRef._evaluate, and if the evaluation is successful, the result is cached under the ForwardRef.__forward_value__ attribute. Any future call to _evaluate will return this cached value.

Coming back to our SafetySettingsType alias, it turns out that it is used in other models (before VertexAIEmbeddings was built), where a reference to google.cloud.aiplatform_v1beta1.types.content.SafetySetting was available [^1]. We end up calling _evaluate for the ForwardRef('SafetySetting') inside List["SafetySetting"], and the result is cached on the instance.

Later on, SafetySettingsType is referenced in the VertexAIEmbeddings model, and the cached value is used.

[^1]: To evaluate forward references, we use what we call a namespace: a dictionary mapping strings to actual objects. The 2.10 refactor is touching this namespace management. What must have happened is that google.cloud.aiplatform_v1beta1.types.content.SafetySetting was (wrongfully or not) included in a namespace in 2.9 while it isn't in 2.10.

yeesian commented 3 days ago

Thank you for the detailed explanation, created a new issue for it in https://github.com/langchain-ai/langchain-google/issues/613

langchain-ai / langchain-google

Import of ChatVertexAI fails with "name 'SafetySetting' is not defined" #610

Explanation of what should happen

What really happens in Pydantic 2.9

module2.py

Model failed to build (calling `model_rebuild()` will raise)

Why the wrong type was used

langchain-ai / langchain-google

Import of ChatVertexAI fails with "name 'SafetySetting' is not defined" #610

Explanation of what should happen

What really happens in Pydantic 2.9

module2.py

Model failed to build (calling model_rebuild() will raise)

Why the wrong type was used

Model failed to build (calling `model_rebuild()` will raise)