Environment Variable Overrides Init Arguments in BaseSettings with Aliases

[imanshafiei540]

Python version: 3.11.8 (arm64) Pydantic version: 2.7.1 pydantic-settings version: 2.2.1 pydantic-core version: 2.18.2

Issue Description:

I'm using Pydantic v2 with BaseSettings (pydantic-settings) to load configurations from environment variables. However, I encountered an issue where environment variables seem to override the initialization arguments, even when I expect the init arguments to take priority.

Here’s a simplified version of my code:

from pydantic import Field, StrictStr
from pydantic_settings import BaseSettings, SettingsConfigDict

class MySettings(BaseSettings):
    api_key: StrictStr = Field(..., alias="TEST_API_KEY")
    aws_region: StrictStr = Field(..., alias="AWS_REGION")

    model_config = SettingsConfigDict(extra="allow", populate_by_name=True)

print(MySettings().model_dump())
print(MySettings(api_key="ANOTHER_API_KEY_TO_OVERRIDE").model_dump())
print(MySettings(api_key=111).model_dump())

{'api_key': 'TEST_API_KEY_VALUE', 'aws_region': 'us-east-1'}
{'api_key': 'ANOTHER_API_KEY_TO_OVERRIDE', 'aws_region': 'us-east-1'}
{'api_key': 111, 'aws_region': 'us-east-1'}

As you can see, there's no validation if I pass an integer to api_key.

Expected Behavior:

Initialization arguments should take priority over environment variables.
Validation should be enforced even with extra="allow".

With model_config = SettingsConfigDict(extra="ignore", populate_by_name=True)":

print(MySettings().model_dump())
# Expected: {'api_key': 'TEST_API_KEY_VALUE', 'aws_region': 'us-east-1'}
# Actual: {'api_key': 'TEST_API_KEY_VALUE', 'aws_region': 'us-east-1'}

print(MySettings(api_key="ANOTHER_API_KEY_TO_OVERRIDE").model_dump())
# Expected: {'api_key': 'ANOTHER_API_KEY_TO_OVERRIDE', 'aws_region': 'us-east-1'}
# Actual: {'api_key': 'TEST_API_KEY_VALUE', 'aws_region': 'us-east-1'}

print(MySettings(api_key=111).model_dump())
# Expected: ValidationError
# Actual: {'api_key': 'TEST_API_KEY_VALUE', 'aws_region': 'us-east-1'}

Also, I posted this as a question in StackOverflow (https://stackoverflow.com/questions/78531095/pydantic-settings-environment-variables-prioritize-over-init-args-with-aliases) and based on the comments there:

By removing the extra attributes (model_config = SettingsConfigDict(populate_by_name=True)):

api_key
  Extra inputs are not permitted [type=extra_forbidden, input_value='ANOTHER_API_KEY_TO_OVERRIDE', input_type=str]

What I expected is that populate_by_name should override that behavior.

Please let me know and I'll provide more details. I'm not sure if I'm missing something here and/or it's an issue with pydantic-settings.

Thank you!

[hramezani]

Thanks @imanshafiei540 for reporting this. Actually, this is a problem in pydantic not in pydantic-settings. pydantic-settings only collects values from different sources and the actual validation happens in pydantic.

So, in your provided example, the collected values for print(MySettings(api_key=111).model_dump()) will be {'TEST_API_KEY': 'TEST_API_KEY_VALUE', 'AWS_REGION': 'us-east-1', 'api_key': 111}

The above values will be passed to pydantic for validation.

This is reproducible by a minimal example in pydantic:

class B(BaseModel):
    foo: str = Field(alias='foo_alias')

    model_config = SettingsConfigDict(extra="allow", populate_by_name=True)

print(B(foo=1, foo_alias='test').model_dump())

Then output will be {'foo': 1}

we can close this issue here and you can create a new issue on pydantic if you want.

[imanshafiei540]

Thanks, @hramezani, for replying back. I've posted a new issue on the Pydantic page (https://github.com/pydantic/pydantic/issues/9516). Let me know if there is something else I can provide.

[hramezani]

Probably it is better to not mention pydantic-setting on the new issue and update the example with my example because this is pydantic issue.

[imanshafiei540]

Correct, it was a bit complicated and hard to understand. I've updated the question based on our discussion here. Thanks!

[hramezani]

@imanshafiei540 I closed this issue in favor of the issue you created on Pydantic.