1Password / onepassword-sdk-python

https://developer.1password.com/docs/sdks/
MIT License
83 stars 8 forks source link

items.get throws PydanticSchemaGenerationError #108

Closed gregorias closed 1 month ago

gregorias commented 1 month ago

Scenario & Reproduction Steps

I have an item that looks like this:

Screenshot 2024-09-08 at 18 31 12

And I try to fetch it with op_client.items.get(vault_id, item_id).

Actual Behavior

The get function throws:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 41, in ruc
  File "/Users/grzesiek/.pyenv/versions/3.12.4/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/onepassword/items.py", line 49, in get
    return Item.model_validate_json(response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/main.py", line 624, in model_validate_json
    return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_mock_val_ser.py", line 93, in __getattr__
    val_ser = self._attempt_rebuild()
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_mock_val_ser.py", line 126, in handler
    if cls.model_rebuild(raise_errors=False, _parent_namespace_depth=5) is not False:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/main.py", line 562, in model_rebuild
    return _model_construction.complete_model_class(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_model_construction.py", line 568, in complete_model_class
    schema = cls.__get_pydantic_core_schema__(cls, handler)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/main.py", line 670, in __get_pydantic_core_schema__
    return handler(source)
           ^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 655, in generate_schema
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 924, in _generate_schema_inner
    return self._model_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 739, in _model_schema
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1115, in _generate_md_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1308, in _common_field_schema
    schema = self._apply_annotations(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2104, in _apply_annotations
    schema = get_inner_schema(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2085, in inner_handler
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 929, in _generate_schema_inner
    return self.match_type(obj)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1029, in match_type
    return self._match_generic_type(obj, origin)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1062, in _match_generic_type
    return self._list_schema(self._get_first_arg_or_any(obj))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 431, in _list_schema
    return core_schema.list_schema(self.generate_schema(items_type))
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 650, in generate_schema
    from_property = self._generate_schema_from_property(obj, obj)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 821, in _generate_schema_from_property
    schema = get_schema(
             ^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/main.py", line 670, in __get_pydantic_core_schema__
    return handler(source)
           ^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 924, in _generate_schema_inner
    return self._model_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 739, in _model_schema
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1115, in _generate_md_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1308, in _common_field_schema
    schema = self._apply_annotations(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2104, in _apply_annotations
    schema = get_inner_schema(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2085, in inner_handler
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 929, in _generate_schema_inner
    return self.match_type(obj)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1038, in match_type
    return self._unknown_type_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 558, in _unknown_type_schema
    raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for [<class 'onepassword.types.ItemFieldDetailsOtp'>]. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.9/u/schema-for-unknown-type

Expected Behavior

The SDK should return the relevant item or at least an exception that is relevant to the caller (e.g., that the functionality is not implemented).

SDK version

0.1.1

Additional information

No response

Marton6 commented 1 month ago

Hey @gregorias ! Thanks for reaching out about this bug. The functionality you're using should be supported and this is indeed a bug.

We will look into it!

AndyTitu commented 1 month ago

Hey @gregorias, could you share more about the project setup in which this error occurs? We're attempting to reproduce this issue and we were wondering whether:

gregorias commented 1 month ago

So while working the questions, I made the issue disappear by specifying v0.1.1 specifically in the manifest file:

onepassword = { git = "https://github.com/1Password/onepassword-sdk-python.git", tag = 'v0.1.1' }

It was like this before:

onepassword = { git = "https://github.com/1Password/onepassword-sdk-python.git" }

source

and it was installing https://github.com/1Password/onepassword-sdk-python/commit/d9e45c99ba4475dcba6369c412f5d705a17bfc43. https://github.com/1Password/onepassword-sdk-python/commit/713e09f287bdb0661514aab82e5673839dfeb845, on the other hand, works fine.
I bisected the commits and https://github.com/1Password/onepassword-sdk-python/commit/d9e45c99ba4475dcba6369c412f5d705a17bfc43 is the bad commit that breaks my use-case.

Here’s a script that I am using to reproduce this issue:

# repro.py
from onepassword.client import Client  # type: ignore

async def get_vault_id(client, vault_name: str) -> str | None:
    async for vault in await client.vaults.list_all():
        if vault.title == vault_name:
            return vault.id
    return None

async def get_item_id(client, vault_id, item_name: str) -> str | None:
    async for item in await client.items.list_all(vault_id):
        if item.title == item_name:
            return item.id
    return None

async def main():
    client = await Client.authenticate(
        auth=
        'redacted_for_obvious_reasons',
        integration_name="Findata Fetcher",
        integration_version="1.0.0")
    af_vault_id = await get_vault_id(client, 'Redacted')
    af_item_id = await get_item_id(client, af_vault_id, 'degiro.nl')
    item = await client.items.get(af_vault_id, af_item_id)
    return client, af_vault_id, af_item_id, item

if __name__ == '__main__':
    import asyncio
    asyncio.run(main())

I am running this in the virtual environment created for Findata Fetcher by Poetry.

I added a print to items.py and here’s the JSON response that the code in the stack trace is seeing (with redactions):

{
    "id": "redacted",
    "title": "degiro.nl",
    "category": "Login",
    "vaultId": "redacted",
    "fields": [
        {
            "id": "username",
            "title": "username",
            "sectionId": null,
            "fieldType": "Text",
            "value": "redacted",
            "details": null
        },
        {
            "id": "password",
            "title": "password",
            "sectionId": null,
            "fieldType": "Concealed",
            "value": "redacted",
            "details": null
        },
        {
            "id": "TOTP_redacted",
            "title": "one-time password",
            "sectionId": "add more",
            "fieldType": "Totp",
            "value": "redacted",
            "details": {
                "type": "Otp",
                "content": {
                    "code": "redacted",
                    "errorMessage": null
                }
            }
        },
        {
            "id": "redacted",
            "title": "mobile app code",
            "sectionId": "add more",
            "fieldType": "Concealed",
            "value": "redacted",
            "details": null
        }
    ],
    "sections": [
        {
            "id": "add more",
            "title": ""
        }
    ],
    "version": 4
}

Looks like details is optional after all.

I’m using Pydantic at 2.9.0, but I can also reproduce it at 2.9.1.

MOmarMiraj commented 1 month ago

Hi @gregorias,

Thanks for looking into it, the optional keyword on the details field was removed on the commits which is why you were getting this error.

This has been fixed and you shouldn't see this error anymore. If it comes up again, please feel free to re-open this issue.

Thanks!