pydantic / pydantic-settings

Settings management using pydantic
https://docs.pydantic.dev/latest/usage/pydantic_settings/
MIT License
502 stars 47 forks source link

Python 3.9 (and earlier I guess), and non PEP-484 type definitions with injector #298

Closed Guibod closed 1 month ago

Guibod commented 1 month ago

I’ve stumbled on an issue on my tox tests will running my tests today. My project relies on injector which relies on typing to infer variables to be injected.

TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

There was also another instance of error about Generics, this was the improper Unions.

I managed to make everything to pass when I updated all occurences of piped type definitions to proper Optional and Union

Screenshot 2024-06-03 at 22 08 31

The bug was detected on 3.9.16, then reproduced in 3.9.19

The stackstrace

I’ve cut everything related to my own project. But it seems that injector requests typing (installed via brew on my mac) to explore the BaseSettings class, and fail. I’ll try to run a minimal project to demonstrate the issue using only get_type_hints on BaseSettings class.

../src/mightstone/app.py:76: in mongo_database
    if self.container.get(MainSettings).storage.implementation == DbImplem.FAKE:
../.venv/lib/python3.9/site-packages/injector/__init__.py:91: in wrapper
    return function(*args, **kwargs)
../.venv/lib/python3.9/site-packages/injector/__init__.py:974: in get
    provider_instance = scope_instance.get(interface, binding.provider)
../.venv/lib/python3.9/site-packages/injector/__init__.py:91: in wrapper
    return function(*args, **kwargs)
../.venv/lib/python3.9/site-packages/injector/__init__.py:800: in get
    instance = self._get_instance(key, provider, self.injector)
../.venv/lib/python3.9/site-packages/injector/__init__.py:811: in _get_instance
    return provider.get(injector)
../.venv/lib/python3.9/site-packages/injector/__init__.py:264: in get
    return injector.create_object(self._cls)
../.venv/lib/python3.9/site-packages/injector/__init__.py:1002: in create_object
    reraise(e, CallError(instance, init_function, (), additional_kwargs, e, self._stack))
../.venv/lib/python3.9/site-packages/injector/__init__.py:190: in reraise
    raise exception.with_traceback(tb)
../.venv/lib/python3.9/site-packages/injector/__init__.py:998: in create_object
    self.call_with_injection(init, self_=instance, kwargs=additional_kwargs)
../.venv/lib/python3.9/site-packages/injector/__init__.py:1020: in call_with_injection
    bindings = get_bindings(callable)
../.venv/lib/python3.9/site-packages/injector/__init__.py:1161: in get_bindings
    type_hints = get_type_hints(callable, include_extras=True)
/opt/homebrew/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py:1497: in get_type_hints
    value = _eval_type(value, globalns, localns)
/opt/homebrew/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py:292: in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
/opt/homebrew/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py:554: in _evaluate
    eval(self.__forward_code__, globalns, localns),
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

<string>:1: TypeError
Guibod commented 1 month ago

The problem is reproduced in this repository: https://github.com/Guibod/pydantic-settings-bug-298

The problem occurs with python 3.8 and 3.9, and only under the influence of injector.

It’s ok to close the bug if you consider that it is not bound to your code. Nonetheless, the PEP-484 trick worked, but PEP-604 seems not to work.

Guibod commented 1 month ago

From my discoveries, the issue is bound to the BaseSettings constructor as inferred by injector. If i define a custom factory:

class Configuration(Module):
    def configure(self, binder: Binder):
        binder.bind(MainSettings, scope=SingletonScope, to=lambda: MainSettings())

Everything runs smoothly.

Guibod commented 1 month ago

I guess you can try to dig in the support for cls.__new__(cls) pattern in your tests.

Screenshot 2024-06-03 at 23 18 17
hramezani commented 1 month ago

Thanks @Guibod for reporting this and preparing the test project.

As you mentioned this is not a problem of pydantic-settings. We want to keep the pipe-based field type because we are happy with that.

Guibod commented 1 month ago

ok, then i’ll push this to injector.