search

pydantic / pydantic-settings

Settings management using pydantic
https://docs.pydantic.dev/latest/usage/pydantic_settings/
MIT License
591 stars 61 forks source link

Error when using validator in nested model with case_sensitive=True and using upper case #308

Closed nstonic closed 3 months ago

nstonic commented 3 months ago

Example:

class DjangoSettings(BaseModel):
    ALLOWED_HOSTS: list[str] = ["127.0.0.1", "localhost"]

    @field_validator("ALLOWED_HOSTS", mode="before")
    def parse_comma_separated(cls, value):
        if not isinstance(value, str):
            return value
        return [item.strip() for item in value.split(",") if item]

class EnvSettings(BaseSettings):
    DJ: DjangoSettings

    model_config = SettingsConfigDict(
        env_nested_delimiter="__",
        case_sensitive=True,
        extra="forbid",
    )

In envs: DJ__ALLOWED_HOSTS: '127.0.0.1, localhost'

Crashes with an error:

Traceback (most recent call last):
  File "/opt/app/src/manage.py", line 22, in <module>
    main()
  File "/opt/app/src/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/app/src/webapp/settings.py", line 20, in <module>
    ENV = EnvSettings()
          ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic_settings/main.py", line 141, in __init__
    **__pydantic_self__._settings_build_values(
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic_settings/main.py", line 311, in _settings_build_values
    return deep_update(*reversed([source() for source in sources]))
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic_settings/main.py", line 311, in <listcomp>
    return deep_update(*reversed([source() for source in sources]))
                                  ^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic_settings/sources.py", line 377, in __call__
    raise SettingsError(
pydantic_settings.sources.SettingsError: error parsing value for field "DJ" from source "EnvSettingsSource"
hramezani commented 3 months ago

Thanks @nstonic for reporting this.

As the ALLOWED_HOSTS field type is list[str], pydantic-settings considers this field as a complex field and parses the value as JSON. So, you can have your desired behavior by removing the field validator and passing the env value as a valid JSON. 

class DjangoSettings(BaseModel):
    ALLOWED_HOSTS: list[str] = ["127.0.0.1", "localhost"]

class EnvSettings(BaseSettings):
    DJ: DjangoSettings

    model_config = SettingsConfigDict(
        env_nested_delimiter="__",
        case_sensitive=True,
        extra="forbid",
    )

Env: DJ__ALLOWED_HOSTS:'["127.0.0.1","localhost"]'

nstonic commented 3 months ago

@hramezani Thank you. I'll keep that in mind. But that's not the problem. This code crashes at any attempt to use field_validator

hramezani commented 3 months ago

Which code? The one that I provided? What error do you get?

nstonic commented 3 months ago

Your code works if case_sensitive=False, but it crashes with ValidationError if case_sensitive=True


  Input should be a valid list [type=list_type, input_value='["127.0.0.1","localhost"]', input_type=str]```
hramezani commented 3 months ago

@nstonic I created https://github.com/pydantic/pydantic-settings/pull/309 to fix the problem.

Please confirm that the fix works.

nstonic commented 3 months ago

Confirmed! Thank you!

hramezani commented 3 months ago

the fix has been released in new pydantic-settings 2.3.2