Closed dvir-ms closed 1 month ago
I'm not familiar with Microsoft KeyVault, but if it is what I would assume it is from the name, this seems like it might be a good candidate for a new SettingsSource in pydantic-settings. This recently-opened PR seems closely related https://github.com/pydantic/pydantic-settings/pull/140, and might serve as a good reference if you wanted to open a PR on pydantic-settings to add support.
Ah cool, I'm working on introducing keyring
which supports Windows Credential Locker. Microsoft Azure KeyVault is a cloud service more like AWS SSM, the keyring
library is local (its repo has no hits for the term 'KeyVault' so I don't think it'd be possible to access the secrets through there). My PR is still work in progress! :slightly_smiling_face:
Can we make progress on this @samuelcolvin? I can lend a hand.
To use Azure Key Vault from localhost, your Microsoft account needs a role, for example, Key Vault Administrator
, and then log in to Azure with az login
. In an App Service, you only assign the role to it.
This is the code I use to read from Azure Key Vault:
application_settings.py
class ApplicationSettings(BaseSettings):
model_config = SettingsConfigDict(
env_nested_delimiter="__",
extra="ignore"
)
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
env_settings,
dotenv_settings,
AzureKeyVaultSettingsSource(settings_cls, os.environ["KEY_VAULT__URL"]),
)
SQL_SERVER__PASSWORD: str
azure_key_vault_settings_source.py
from typing import Any, Optional
from azure.core.exceptions import ResourceNotFoundError
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
from pydantic.fields import FieldInfo
from pydantic_settings import (
BaseSettings,
PydanticBaseSettingsSource,
)
class AzureKeyVaultSettingsSource(PydanticBaseSettingsSource):
_credential: DefaultAzureCredential
_secret_client: SecretClient
def __init__(self, settings_cls: type[BaseSettings], url: str) -> None:
self._credential = DefaultAzureCredential()
self._secret_client = SecretClient(vault_url=url, credential=self._credential)
super().__init__(settings_cls)
def get_field_value(
self, field: FieldInfo, field_name: str
) -> tuple[Any, str, bool]:
field_value: Optional[Any] = None
# It's not possible to use underscores in Azure Key Vault
secret_name = field_name.replace("_", "-")
try:
secret = self._secret_client.get_secret(secret_name) # type: ignore
field_value = secret.value
except ResourceNotFoundError:
field_value = None
return field_value, field_name, False
def prepare_field_value(
self, field_name: str, field: FieldInfo, value: Any, value_is_complex: bool
) -> Any:
return value
def __call__(self) -> dict[str, Any]:
data: dict[str, Any] = {}
for field_name, field in self.settings_cls.model_fields.items():
field_value, field_key, value_is_complex = self.get_field_value(
field, field_name
)
field_value = self.prepare_field_value(
field_name, field, field_value, value_is_complex
)
if field_value is not None:
data[field_key] = field_value
return data
Python packages: azure-keyvault-secrets
and azure-identity
.
Thanks @AndreuCodina for the settings source code. You can make a PR for this if you would like. Please consider to add proper test and documentation for this
Initial Checks
Description
Hi
Currently, we use the Pydantic with a wrapper that checks if the field exists (in addition to the env var) in the KeyVault.
This is not ideal as it is done in a separate place.
A new feature of fetching data from KeyVault will be very helpful.
Thanks!
Affected Components
.model_dump()
and.model_dump_json()
model_construct()
, pickling, private attributes, ORM modeSelected Assignee: @Kludex
Selected Assignee: @samuelcolvin