.env file values take precedence over environment variables in BaseSettings

[gerardpc]

According to the documentation, variables defined in the environment take precedence over variables in .env files when instantiating a BaseSettings object:

"Even when using a dotenv file, pydantic will still read environment variables as well as the dotenv file, environment variables will always take priority over values loaded from a dotenv file."

However, if I define the following .env_pyd file

SOMEPREFIX_PREFIX="d"
SOMEPREFIX_ENV="e"

and load it with

import os
from pydantic_settings import BaseSettings, SettingsConfigDict

from devtools import debug

# this settings object ignores the env file
class SettingsNoEnvFile(BaseSettings):
    PREFIX: str = "a"
    ENV: str = "b"

# this settings object loads variables in .env_pyd file
class Settings(BaseSettings):
    PREFIX: str = "a"
    ENV: str = "b"
    model_config = SettingsConfigDict(
        frozen=True,
        env_file=".env_pyd",
        env_prefix="SOMEPREFIX_",
    )

# set and print env vars
os.environ["PREFIX"] = "c"
os.environ["ENV"] = "d"

# instantiate and print the settings
settings_no_env = SettingsNoEnvFile()
settings = Settings()

debug(settings_no_env)
debug(settings)

The output is:

  settings_no_env: SettingsNoEnvFile(
      PREFIX='c',
      ENV='d',
  ) (SettingsNoEnvFile)

  settings: Settings(
      PREFIX='d',
      ENV='e',
  ) (Settings)

i.e., the inverse order of preference. Is this the expected behaviour and I didn't understand the documentation, or is that a bug?

[hramezani]

Thanks @gerardpc for reporting this.

You defined env_prefix="SOMEPREFIX_" in your settings model but your environment variables name don't start with SOMEPREFIX_, So, pydantic-settings will ignore them. Your problem will be fixed by adding SOMEPREFIX_ to your environment variables name

[gerardpc]

Thanks @hramezani, yes it works as you say. I was expecting that the env_prefix in

model_config = SettingsConfigDict(
    frozen=True,
    env_file=".env_pyd",
    env_prefix="SOMEPREFIX_",
)

would only affect the variables from the env file.

[palunel]

@hramezani but if I set the environment variable to something like:

ENV: str = Field('prod', alias='ENV')

My understanding is that it will always look for an environment variable named ENV regardless of what the env_prefix prefix is set to. So I should be able to os.environ['STAGE'] = 'ENV'? However, this does not seem to work if I set STAGE with export STAGE=DEV and the .env overrides the OS env. Unless it is perhaps an issue with how the OS env variable is set, however, neither using export ENV=DEV nor setting it directly in ~/.zshrc seems to result in it being read by Pydantic.

[hramezani]

@palunel Could you please create a new issue with a small example to reproduce the problem?