Env does not override nested value when other sources are used

[rusmux]

In this example, If I use YAML source, I cannot override sub_value field with an environment variable.

In main.py:

from pydantic import Field
from pydantic_settings import (
    BaseSettings,
    PydanticBaseSettingsSource,
    SettingsConfigDict,
    YamlConfigSettingsSource,
)

class PydanticBaseSettings(BaseSettings):

    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        return (init_settings, env_settings, dotenv_settings,
                YamlConfigSettingsSource(settings_cls), file_secret_settings)

class SubSettings(BaseSettings):
    model_config = SettingsConfigDict(env_prefix="sub_")
    value: str

class Settings(PydanticBaseSettings):
    model_config = SettingsConfigDict(yaml_file="config.yaml")

    value: str
    sub: SubSettings = Field(default_factory=SubSettings)

print(Settings())

In config.yaml:

value: a

sub:
  value: a

In terminal: value=b sub_value=b python -m main

Gives:

value='b' sub=SubSettings(value='a')

The first value is overridden, but the second is not. If I do not use YAML source everything works as expected. I tried using PydanticBaseSettings for SubSettings, but it doesn't help.

[hramezani]

Thanks @rusmux for reporting this.

Your example code has some problems:

• You have to use BaseModel instead of BaseSettings in sub-models https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values
• The env_prefix in sub-model does not have any effect, because pydantic-settings only considers the config of root model
• Better to use env_nested_delimiter config for sub models

So, at the end you can collect values like you want by changing your code to:

from pydantic import Field, BaseModel
from pydantic_settings import (
    BaseSettings,
    PydanticBaseSettingsSource,
    SettingsConfigDict,
    YamlConfigSettingsSource,
)

class PydanticBaseSettings(BaseSettings):

    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        return (init_settings, env_settings, dotenv_settings,
                YamlConfigSettingsSource(settings_cls), file_secret_settings)

class SubSettings(BaseModel):
    value: str

class Settings(PydanticBaseSettings):
    model_config = SettingsConfigDict(yaml_file="config.yaml", env_nested_delimiter="__")

    value: str
    sub: SubSettings

print(Settings())

You need to call your script by value=b sub__value=c python ...

[rusmux]

I see, thank you!