search

python-attrs / cattrs

Composable custom class converters for attrs, dataclasses and friends.
https://catt.rs
MIT License
791 stars 110 forks source link

RecursionError on recursive nested class #409

Open philpep opened 1 year ago

philpep commented 1 year ago

Description

Hi, I've RecursionError raised when structuring nested class.

What I Did

Here's a minimal code triggering the issue:

from __future__ import annotations

import attrs
import cattrs

@attrs.frozen
class Source:
    child: dict[str, Source]

cattrs.structure({"child": {}}, Source)

And part of the traceback:

Traceback (most recent call last):
  File "t.py", line 12, in <module>
    cattrs.structure({"child": {}}, Source)
[...]
  File "lib/python3.11/site-packages/cattrs/converters.py", line 1043, in gen_structure_mapping
    h = make_mapping_structure_fn(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/gen/__init__.py", line 679, in make_mapping_structure_fn
    val_handler = converter._structure_func.dispatch(val_type)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/dispatch.py", line 48, in _dispatch
    res = self._function_dispatch.dispatch(typ)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/dispatch.py", line 133, in dispatch
    return handler(typ)
           ^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/converters.py", line 985, in gen_structure_attrs_fromdict
    h = make_dict_structure_fn(
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/gen/__init__.py", line 311, in make_dict_structure_fn
    handler = find_structure_handler(
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/gen/_shared.py", line 47, in find_structure_handler
    handler = c._structure_func.dispatch(type)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/dispatch.py", line 48, in _dispatch
    res = self._function_dispatch.dispatch(typ)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/dispatch.py", line 133, in dispatch
    return handler(typ)
           ^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/converters.py", line 1043, in gen_structure_mapping
    h = make_mapping_structure_fn(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/gen/__init__.py", line 679, in make_mapping_structure_fn
    val_handler = converter._structure_func.dispatch(val_type)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/dispatch.py", line 48, in _dispatch
    res = self._function_dispatch.dispatch(typ)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/dispatch.py", line 133, in dispatch
    return handler(typ)
           ^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/converters.py", line 985, in gen_structure_attrs_fromdict
    h = make_dict_structure_fn(
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/gen/__init__.py", line 302, in make_dict_structure_fn
    t = deep_copy_with(t, mapping)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/_generics.py", line 14, in deep_copy_with
    tuple(
  File "lib/python3.11/site-packages/cattrs/_generics.py", line 17, in <genexpr>
    else (deep_copy_with(a, mapping) if is_generic(a) else a)
                                        ^^^^^^^^^^^^^
  File "lib/python3.11/site-packages/cattrs/_compat.py", line 497, in is_generic
    or is_subclass(obj, Generic)
       ^^^^^^^^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded
PIG208 commented 1 year ago

201 and #299 are related. I think a workaround would be using ForwardRef or a NewType hook.

Tinche commented 1 year ago

We need to improve make_dict_structure_fn to take into account classes we're already in the process of generating (or just catch the RecursionError, but that's less efficient). #299 is just because we don't support typing.Self I think.

gpalmer-latai commented 3 months ago

Hi,

Does a workaround / solution exist for this yet?

I think a workaround would be using ForwardRef or a NewType hook.

Is there an example of how to do this anywhere?

gpalmer-latai commented 3 months ago

I ended up coming up with this workaround:

@attrs.define
class Foo:
  nested_dict: dict[str, Foo] = attrs.field(factory=dict)
    converter = cattrs.Converter()

    # Overriding the deserialization of this type is necessary to prevent an infinite recursion bug:
    # https://github.com/python-attrs/cattrs/issues/409
    def structure_nested_dict(
        nested_dict: dict[str, Any], _: type[dict[str, Foo]]
    ) -> dict[str, Foo]:
        return {key: converter.structure(val, Foo) for key, val in nested_dict.items()}

    converter.register_structure_hook(
        Foo,
        cattrs.gen.make_dict_structure_fn(
            Foo,
            converter,
            nested_dict=cattrs.gen.override(struct_hook=structure_nested_dict),
        ),
    )