Validation error for 3 levels of nested dicts in v2.3.0

bpicardat commented 4 months ago

Hello, My model does not work anymore with the latest version of pydantic-settings. Here is a test that reproduces my issue (env is the fixture from the pydantic-settings tests):

def test_nested_dicts(env):
    class Settings(BaseSettings):
        nested: Dict[str, Dict[str, Dict[str, str]]]

        model_config = SettingsConfigDict(env_nested_delimiter='__')

    env.set('nested__foo__a__b', 'bar')
    s = Settings()
    assert s.model_dump() == {'nested': {'foo': {'a': {'b': 'bar'}}}}

This test passes in version 2.2.1 of pydantic-settings but fails in version 2.3.0 with error:

E               pydantic_settings.sources.SettingsError: error parsing value for field "nested" from source "EnvSettingsSource"

.venv/lib/python3.11/site-packages/pydantic_settings/sources.py:377: SettingsError

This happens with Python 3.11.9 on Linux, and the following packages:

Package           Version
----------------- -------
annotated-types   0.7.0
iniconfig         2.0.0
packaging         24.0
pip               24.0
pluggy            1.5.0
pydantic          2.7.3
pydantic_core     2.18.4
pydantic-settings 2.3.0
pytest            8.2.2
python-dotenv     1.0.1
setuptools        69.5.1
typing_extensions 4.12.1
wheel             0.43.0

Is this considered a regression or should I find an alternative way to solve my issue?

xaniasd commented 4 months ago

chiming in on this if I may, the following fails with the same error when I set nested__test__val=myval. I think it doesn't have to do with the number of nesting levels, but generally setting values in dicts. I suspect these lines, i.e. it's trying to parse a non-JSON value as such. This snippet does work with pydantic-settings < 2.3.0

from pydantic import BaseModel
from pydantic_settings import BaseSettings, SettingsConfigDict

class NestedModel(BaseModel):
    val: str

class Settings(BaseSettings):
    val: str
    nested: dict[str, NestedModel]
    model_config = SettingsConfigDict(
        env_nested_delimiter="__",
    )

if __name__ == "__main__":
    Settings(val="test")