BaseSettings fails to load nested models overridden by ENV with case_senitivity=False

[chuckyblack]

How to reproduce:

requirements.txt

pydantic==2.7.0
pydantic-settings==2.2.1

config.py

import os
from pydantic import BaseModel
from pydantic_settings import BaseSettings, SettingsConfigDict

os.environ['TEST_nested_nestedNested_var1'] = 'test'

class NestedNested(BaseModel):
    var1: str

class Nested(BaseModel):
    nestedNested: NestedNested

class Config(BaseSettings):
    model_config = SettingsConfigDict(
        case_sensitive=False,
        env_prefix='TEST_',
        env_nested_delimiter='_'
    )

    # optional, it should be None if no env is set
    nested: Nested | None = None

if __name__ == '__main__':
    config = Config()

This throws error:

Traceback (most recent call last):
  File "/pydanticTest/src/config.py", line 28, in <module>
    config = Config()
             ^^^^^^^^
  File "/pydanticTest/venv/lib/python3.12/site-packages/pydantic_settings/main.py", line 84, in __init__
    super().__init__(
  File "/pydanticTest/venv/lib/python3.12/site-packages/pydantic/main.py", line 175, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Config
nested.nestedNested
  Field required [type=missing, input_value={'nestednested': {'var1': 'test'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing

Process finished with exit code 1

If I change case_sensitivity=True, it works fine.

[hramezani]

Thanks for reporting this.

pydantic-settings convert all the env variables to lowercase if case_sensitivity=False(the default). that's why it can't find value for the field nestedNested.

This was the default behavior on V1 and we keep it on V2. So, we can't change it in V2

[chuckyblack]

Is there any reason why this can't be fixed?

What about solution where the field name nestedNested would be coverted to lowercase too during matching?

[hramezani]

Is there any reason why this can't be fixed?

I think it needs a big change in pydantic-settings and it can't be done in V2 because it will introduce some breaking changes.

What about solution where the field name nestedNested would be coverted to lowercase too during matching?

This is also happening. That's why you see the lowercase fieldname in error input_value={'nestednested': {'var1': 'test'}} but pydantic needs the real field name nestedNested

[hramezani]

@chuckyblack
I created https://github.com/pydantic/pydantic-settings/pull/294 to fix the problem. Could you please confirm it?

[chuckyblack]

Great, it's working. @hramezani Could you release a new version, please?

[hramezani]

This will be included in the new release. I am trying to prepare a new release in couple of days