Support Pydantic Defaults

[andysnowden]

The Pydantic Field object allows the specification of a default value. However, when used with this plugin the default is ignored and an exception is returned. Is this just a matter of configuration/implementation or does the plugin need to add support for this usecase?

import os

from pydantic import BaseSettings, Field
from pydantic_vault import vault_config_settings_source
from fastapi.logger import logger

from core.constants import *

class Settings(BaseSettings):
    # Vault Secret Path
    vault_secret_path = os.getenv("VAULT_SECRET_PATH", VAULT_SECRET_PATH)

    CMR_MODEL_VERSION2: str = Field(default="test123", vault_secret_path=vault_secret_path, vault_secret_key="some_vault_key")

    class Config:
        # Support .env files
        env_file = ".env"
        env_file_encoding = "utf-8"

        # Support Vault
        # This will use token and fall back to app-role
        vault_url: str = os.getenv("VAULT_URL", VAULT_URL)
        vault_token: str = os.getenv("VAULT_TOKEN", "")
        vault_role_id: str = os.getenv("VAULT_ROLE_ID", VAULT_ROLE_ID)
        vault_secret_id_id: str = os.getenv("VAULT_SECRET_ID", "")
        vault_secret_mount_point: str = "kv"

        @classmethod
        def customise_sources(
                cls,
                init_settings,
                env_settings,
                file_secret_settings,
        ):
            return (
                init_settings,
                env_settings,
                vault_config_settings_source,
                file_secret_settings
            )

try:
    settings = Settings()
    print("settings": settings)
except:
    logger.error("Failed to parse app settings")

So with the above example code if the vault KV pair does not exist you get a KeyError exception.

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [4396] using watchgod
Process SpawnProcess-1:
Traceback (most recent call last):
....ommited....
  File ".\core\config.py", line 75, in <module>
    raise e
  File ".\core\config.py", line 72, in <module>
    settings = Settings()
  File "pydantic\env_settings.py", line 37, in pydantic.env_settings.BaseSettings.__init__
  File "pydantic\env_settings.py", line 63, in pydantic.env_settings.BaseSettings._build_values
  File "D:\PycharmProjects\ms-datascience\venv\lib\site-packages\pydantic_vault\vault_settings.py", line 148, in vault_config_settings_source
    vault_val = vault_client.secrets.kv.v2.read_secret_version(
KeyError: 'some_vault_key'

However, if the KV pair does exist but is set or even blank it takes that value and ignores the default

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [17000] using watchgod
CMR_MODEL_VERSION2='not my default'
INFO:     Started server process [20268]
INFO:uvicorn.error:Started server process [20268]
INFO:     Waiting for application startup.
INFO:uvicorn.error:Waiting for application startup.
INFO:     Application startup complete.
INFO:uvicorn.error:Application startup complete.

Is this just an edge case that is not supported by the plugin? If there was a way to selectively tell the plugin to use defaults if there was a vault exception that would be fine too.

[nymous]

Hello @andysnowden! Thank you for your this issue, it feels weird to have feedback even though the code is very much in alpha ^^' You are right about this, I have quickly debugged your example and it is an error on my side: whether I find a key in Vault or not I return it in the values dict that Pydantic uses to populate the model, sometimes with a None value. Pydantic then validates this dict and because None is not a valid type, it gets angry ^^

I will see to fix this, or you can do a PR if you want :wink: (if you look at vault_settings.py I try to get the Vault key, log a message if it is not found, but I add it to the dict in any case; this should only happen if we found a value).

[nymous]

Hey @andysnowden! I fixed the issue and published version 0.4.1a1 on PyPI, try it and tell me how it goes!