Default field value priority is not consistent with setting sources priority

[PrieJos]

Dear Pydantic team,

I am using pydantic-settings in one of my projects to implement all the settings load logic. But I ran into the following issue which I think it might be a bug or at least something not consistent with what the documentation says.

So, according to the section Field value priority in the documentation, you can read as follows:

In the case where a value is specified for the same Settings field in multiple ways, the selected value is determined as follows (in descending order of priority):

1. Arguments passed to the Settings class initialiser.
2. Environment variables, e.g. my_prefix_special_function as described above.
3. Variables loaded from a dotenv (.env) file.
4. Variables loaded from the secrets directory.
5. The default field values for the Settings model.

That default priority is perfectly fine for me. However as far as I could check, the following code is not consistent with that:

import os
from pprint import pp
from pathlib import Path

import dotenv
from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict

class Settings(BaseSettings):
    """My settings demo class."""

    foo: str = Field("xxx", alias="MYAPP_FOO")
    bar: int = Field(0, alias='MYAPP_BAR')

    model_config = SettingsConfigDict(
        populate_by_name=True,
        case_sensitive=True, 
        extra="ignore",
        env_file="/tmp/pydantic/.env",
        env_prefix="MYAPP_"
    )

def populate_dotenv(__dotenv: Path, **vars) -> Path:
    __dotenv.parent.mkdir(mode=0o0755, parents=True, exist_ok=True)
    __dotenv.touch(mode=0o0600)
    for var, value in vars.items():
        dotenv.set_key(__dotenv, var, value)
    return __dotenv

# Populate dotenv
dotenv_file = Path("/tmp/pydantic/.env")
populate_dotenv(
    dotenv_file,
    MYAPP_FOO="value-from-dotenv",
    MYAPP_BAR="1",
)
assert dotenv_file.exists()
with dotenv_file.open(mode="r") as fh:
    pp(tuple(line.strip() for line in fh.readlines()))

# Populate OS environment
os.environ["MYAPP_FOO"] = "value-from-osenv"
os.environ["MYAPP_BAR"] = "2"

# Load settings
settings = Settings(foo="value-from-init", bar=3)
pp(settings.model_dump())

# Clean up
del os.environ["MYAPP_FOO"]
del os.environ["MYAPP_BAR"]

While I was expecting to get the following output out of it:

("MYAPP_FOO='value-from-dotenv'", "MYAPP_BAR='1'")
{'foo': 'value-from-init', 'bar': 3}

The truth is that this is not the case. What I'm getting back is:

("MYAPP_FOO='value-from-dotenv'", "MYAPP_BAR='1'")
{'foo': 'value-from-osenv', 'bar': 2}

That means the explicit init arguments I am giving when instantiating Settings object are not taking into account even though I set the configuration parameter populate_by_name=True. However if you replace field name by the alias as follows, then it works as expected:

settings = Settings(MYAPP_FOO="value-from-init", MYAPP_BAR=3)

Did I miss some configuration or is it a real bug?

BTW - The versions that are installed in my python environment are:

• pydantic 2.6.3
• pydantic-settings 2.2.1

Thanks in advanced for your support.

Cheers, Jose M. Prieto

[hramezani]

Thanks @PrieJos for reporting this.

This is the final collected values from sources by pydantic-settings

{'MYAPP_FOO': 'value-from-osenv', 'MYAPP_BAR': '2', 'foo': 'value-from-init', 'bar': 3}

It will be passed to pydantic for validation. As populate_by_name is enabled, pydantic picks the values that you mentioned. There is no strict rule to tell pydantic which value has to be picked when populate_by_name is enabled and values are provided for both field name and alias.

So, you can fix the problem by providing init values by alias name like:

settings = Settings(**{'MYAPP_FOO':"value-from-init", 'MYAPP_BAR':3})

[PrieJos]

Thanks @hramezani for the reply.

So then as far as I understood you, the behavior you described here...

This is the final collected values from sources by pydantic-settings

{'MYAPP_FOO': 'value-from-osenv', 'MYAPP_BAR': '2', 'foo': 'value-from-init', 'bar': 3}

It will be passed to pydantic for validation. As populate_by_name is enabled, pydantic picks the values that you mentioned. There is no strict rule to tell pydantic which value has to be picked when populate_by_name is enabled and values are provided for both field name and alias.

... is somehow not fully consistent with the excerpt of the documentation I pasted in the ticket whenever aliases are used. Or at least, it is not clear enough in my opinion. So maybe a side note on the documentation warning about this will help. 🙏🏻

On the other hand, this solution you provided I checked it already and as you mentioned is returning the expected output.

So, you can fix the problem by providing init values by alias name like:

settings = Settings(**{'MYAPP_FOO':"value-from-init", 'MYAPP_BAR':3})

However I opted to write my own PydanticBaseSettingsSource that replaces the out-of-the-box InitSettingsSource implementation which is basically doing your proposed solution behind the scene.

Thanks!

Cheers Jose M. Prieto

[hramezani]

@PrieJos probably the documentation can be more clear about this. You can make a PR and improve it if you want and we appreciate it. BTW, I am closing the issue for now.