Support _FILE envars from dotenv files

[inean]

Extend pydantic dotenv_settings implementation to also handle _FILE cases presents on dotenv files

[h4l]

Hey, thanks for your interest in this. Could you tell me more about your use case? Are you using Pydantic's built-in .env file support currently?

I wonder if it would be sufficient for you to use https://saurabh-kumar.com/python-dotenv/#getting-started to load your .env file into environment variables before calling Pydantic to load the settings (with this module enabled)? I think that would work with this module as it is today.

[inean]

Hey, thanks for your interest in this. Could you tell me more about your use case? Are you using Pydantic's built-in .env file support currently?

Yes

I wonder if it would be sufficient for you to use https://saurabh-kumar.com/python-dotenv/#getting-started to load your .env file into environment variables before calling Pydantic to load the settings (with this module enabled)? I think that would work with this module as it is today.

It may work. The fact is that it seems that dotenv_settings is unusable with a list of submodels, so, with something like:

class SubModel(BaseModel):
     a_value: str

class Settings(BaseSettings):
     submodels: list[SubModel]

It's not clear how to pass submodels using env vars (SUBMODELS__0__A_VALUE=F00 won't work). I will try use python-dotenv to handle submodels case and remove dotenv_settings from class Settings sources. Thas way we get support to dotenv files and also env vars with _FILE

[inean]

Ok, I finally end injecting get_field_value from your class into EventSettingsSource. That way we get _FILE support also on DotEventSettingsSource:

class _EnvSettingsSource(EnvSettingsSource):
    def get_field_value(
        self, field: FieldInfo, field_name: str
    ) -> tuple[Any, str, bool]:
        # Attenpt to get field wirth default method
        env_val, fkey, is_complex = self._e_get_field_value_(field, field_name)
        # Fallback ro _FILE based get_field_value if previous method fails
        if env_val is None:
            env_val, fkey, is_complex = self._f_get_field_value_(field, field_name)
        # Return the value, key and if the value is complex
        return env_val, fkey, is_complex

EnvSettingsSource._e_get_field_value_ = EnvSettingsSource.get_field_value
EnvSettingsSource._f_get_field_value_ = FileSuffixEnvSettingsSource.get_field_value

I know this is not best practice, but works for my case.

What doy you think about PR to pydantic settings with your get_field_value implementation ? IMHO, with an added entry like 'read_value_from_file=True' in SettingsConfigDict we may maintain current PydanticSettings behaviour and support _FILE for env vars and dotenv files without my solution...

[h4l]

Thanks for following up @inean, glad you could work around like this. It would be nice to have this functionality by default. I suppose it would need to be opt-in to avoid unexpected references to arbitrary files. Perhaps an allow list/glob of dirs, and allow list of vars to enable _FILE support for.

I'll reach out and see if they'd be interested in integrating _FILE support.

Otherwise I'd be happy to add support for the .env version of the class here similar to this. I guess another alternative could be for python-dotenv to support it, as Pydantic uses it to parse .env files.

[h4l]

Reopening to track this.