Unable to Override Environment Variable in Pydantic Settings

[sergenciftci]

When using Pydantic-settings, if an attribute is set by an environment variable, it can not be overridden by passing a value directly during instantiation of the settings class. I would expect MyBaseSettings(foo='somevalue') to work, when using the configuration option populate_by_name=True. It does work, when the environment variable MY_PREFIX_FOO was not set before.

What does work is MyBaseSettings(MY_PREFIX_FOO='somevalue')

from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict
import os

import pydantic_settings
import pydantic

print(pydantic.version.version_info())
"""
             pydantic version: 2.7.2
        pydantic-core version: 2.18.3
          pydantic-core build: profile=release pgo=true
                 install path: /Users/.../lib/python3.11/site-packages/pydantic
               python version: 3.11.9 (main, May 24 2024, 08:43:48) [Clang 15.0.0 (clang-1500.3.9.4)]
                     platform: macOS-14.5-arm64-arm-64bit
             related packages: mypy-1.10.0 fastapi-0.111.0 typing_extensions-4.12.2 pydantic-settings-2.2.1
                       commit: unknown
"""
print(pydantic_settings.version.VERSION)
"""
2.2.1
bar='' local_dev=False foo='FromEnvFoo'
"""

os.environ['MY_PREFIX_FOO'] = 'FromEnvFoo'

class MyBaseSettings(BaseSettings):
    model_config = SettingsConfigDict(validate_assignment=True, env_prefix="MY_PREFIX_", populate_by_name=True)
    bar: str = ""
    local_dev: bool = Field(default=False, validation_alias="LOCAL_DEV")
    foo: str = Field(default='default foo', validation_alias="MY_PREFIX_FOO")

print(MyBaseSettings())
# bar='' local_dev=False foo='FromEnvFoo'
print(MyBaseSettings(foo='somevalue'))
"""
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[2], line 40
     33 print(x)
     34 # x.foo = "1"
     35 # print(x)
     36 # print('-----')
     37 # print(MyBaseSettings(MY_PREFIX_FOO='1'))
     38 
     39 # bar='' local_dev=False foo='FromEnvFoo'
---> 40 print(MyBaseSettings(foo='somevalue'))

File ~/.pyenv/versions/3.11.9/envs/.../lib/python3.11/site-packages/pydantic_settings/main.py:84, in BaseSettings.__init__(__pydantic_self__, _case_sensitive, _env_prefix, _env_file, _env_file_encoding, _env_ignore_empty, _env_nested_delimiter, _env_parse_none_str, _secrets_dir, **values)
     71 def __init__(
     72     __pydantic_self__,
     73     _case_sensitive: bool | None = None,
   (...)
     82 ) -> None:
     83     # Uses something other than `self` the first arg to allow "self" as a settable attribute
---> 84     super().__init__(
     85         **__pydantic_self__._settings_build_values(
     86             values,
     87             _case_sensitive=_case_sensitive,
     88             _env_prefix=_env_prefix,
     89             _env_file=_env_file,
     90             _env_file_encoding=_env_file_encoding,
     91             _env_ignore_empty=_env_ignore_empty,
     92             _env_nested_delimiter=_env_nested_delimiter,
     93             _env_parse_none_str=_env_parse_none_str,
     94             _secrets_dir=_secrets_dir,
     95         )
     96     )

File ~/.pyenv/versions/3.11.9/envs/.../lib/python3.11/site-packages/pydantic/main.py:176, in BaseModel.__init__(self, **data)
    174 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    175 __tracebackhide__ = True
--> 176 self.__pydantic_validator__.validate_python(data, self_instance=self)

ValidationError: 1 validation error for MyBaseSettings
foo
  Extra inputs are not permitted [type=extra_forbidden, input_value='somevalue', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
"""

[hramezani]

Thanks @sergenciftci for reporting this issue.

Actually, It is an issue in Pydantic. pydantic-settings only collects the values from different sources and passes it to Pydantic for validation. here is the Github issue reported in Pydantic.

[sergenciftci]

Thanks @hramezani for the fast reply. I'll close this issue then.