Closed bschoenmaeckers closed 1 month ago
@kschwab Could you please take a look?
@bschoenmaeckers I'm unable to reproduce the issue. Can you paste the versions for pydantic
and pydantic-settings
? The above should be resolved in pydantic-settings v2.3.2
.
Also, the SubSettings
class should inherit BaseModel
and not BaseSettings
. See relevant note in Parsing environment variable values.
Tested it on the latest commit of main:
>python -c "import pydantic; print(pydantic.version.version_info())"
pydantic version: 2.7.0
pydantic-core version: 2.18.1
pydantic-core build: profile=release pgo=true
install path: D:\repo\pydantic-settings\.venv\Lib\site-packages\pydantic
python version: 3.12.4 (tags/v3.12.4:8e8a4ba, Jun 6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)]
platform: Windows-11-10.0.22631-SP0
related packages: typing_extensions-4.11.0
commit: unknown
Also, the
SubSettings
class should inheritBaseModel
and notBaseSettings
. See relevant note in Parsing environment variable values.
My SubSettings has to ingerit from BaseSettings because I want to set custom config values to that sub model only, like secrets_dir
for example.
If you run my example with pytest on the current main then it should crash.
Run pytest test.py
import dataclasses
import os
import pytest
from pydantic_settings import BaseSettings, SettingsConfigDict
class SetEnv:
def __init__(self):
self.envars = set()
def set(self, name, value):
self.envars.add(name)
os.environ[name] = value
def pop(self, name):
self.envars.remove(name)
os.environ.pop(name)
def clear(self):
for n in self.envars:
os.environ.pop(n)
@pytest.fixture
def env():
setenv = SetEnv()
yield setenv
setenv.clear()
def test_nested_dataclass_setting(env):
@dataclasses.dataclass
class DataClass:
value: str
class SubSettings(BaseSettings, DataClass):
model_config = SettingsConfigDict(alias_generator=str.lower)
class Cfg(BaseSettings):
model_config = SettingsConfigDict(env_nested_delimiter='__')
sub: SubSettings
env.set('SUB__VALUE', 'something')
cfg = Cfg()
Thanks @bschoenmaeckers. It was the dataclasses. I was using pydantic dataclasses instead of generic dataclasses.
I was able to reproduce prior to pydantic-settings 2.3.2
. For this case, it should be resolved with pydantic-settings 2.3.2+
. Can you confirm?
However, it does raise a potential issue with either CLI settings or pydantic is_pydantic_dataclass
. @hramezani is the below expected:
import dataclasses
from pydantic import BaseModel
from pydantic.dataclasses import is_pydantic_dataclass
@dataclasses.dataclass
class DataClass:
value: str
class DataModel(BaseModel, DataClass):
...
print(is_pydantic_dataclass(DataModel))
#> True
Thanks @bschoenmaeckers. It was the dataclasses. I was using pydantic dataclasses instead of generic dataclasses.
Yes my bad, it is the vanilla dataclass.
For this case, it should be resolved with pydantic-settings 2.3.2+.
My test passes for version 2.3.2 & 2.3.3 but it fails again for 2.3.4 & 2.4.0
Got it. Yes, the 2.3.4
and 2.4.0
failures are different from the CliSettingsSource
exception:
pydantic_settings/main.py:145: in __init__
**__pydantic_self__._settings_build_values(
pydantic_settings/main.py:330: in _settings_build_values
source_state = source()
pydantic_settings/sources.py:430: in __call__
field_value = self.prepare_field_value(field_name, field, field_value, value_is_complex)
pydantic_settings/sources.py:623: in prepare_field_value
env_val_built = self.explode_env_vars(field_name, field, self.env_vars)
pydantic_settings/sources.py:749: in explode_env_vars
target_field = self.next_field(target_field, last_key, self.case_sensitive)
pydantic_settings/sources.py:700: in next_field
annotation.__pydantic_fields__
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <class 'test_bed.test_nested_dataclass_setting.<locals>.SubSettings'>, item = '__pydantic_fields__'
def __getattr__(self, item: str) -> Any:
"""This is necessary to keep attribute access working for class attribute access."""
private_attributes = self.__dict__.get('__private_attributes__')
if private_attributes and item in private_attributes:
return private_attributes[item]
if item == '__pydantic_core_schema__':
# This means the class didn't get a schema generated for it, likely because there was an undefined reference
maybe_mock_validator = getattr(self, '__pydantic_validator__', None)
if isinstance(maybe_mock_validator, MockValSer):
rebuilt_validator = maybe_mock_validator.rebuild()
if rebuilt_validator is not None:
# In this case, a validator was built, and so `__pydantic_core_schema__` should now be set
return getattr(self, '__pydantic_core_schema__')
> raise AttributeError(item)
E AttributeError: __pydantic_fields__. Did you mean: '__pydantic_fields_set__'?
../.local/lib/python3.12/site-packages/pydantic/_internal/_model_construction.py:242: AttributeError
@hramezani this is in the PydanticBaseEnvSettingsSource
, also related to is_pydantic_dataclass
query from above.
However, it does raise a potential issue with either CLI settings or pydantic is_pydantic_dataclass. @hramezani is the below expected:
@kschwab Yes, dataclasses.is_dataclass(DataModel)
returns True
also.
@hramezani this is in the PydanticBaseEnvSettingsSource, also related to is_pydantic_dataclass query from above.
I've created https://github.com/pydantic/pydantic-settings/pull/357 to fix the problem. Take a look if you have time.
I have a dataclass that I don't control as a nested object (wrapped by a BaseSettings class). This used to work great but after v2.3.0 (#214) it throws a AttributeError. See the following testcase that passes on
<=v2.2.1
but fails on>=v2.3.0
.The exception
```python pydantic_settings\main.py:141: in __init__ **__pydantic_self__._settings_build_values( pydantic_settings\main.py:260: in _settings_build_values CliSettingsSource( pydantic_settings\sources.py:902: in __init__ self._connect_root_parser( pydantic_settings\sources.py:1236: in _connect_root_parser self._add_parser_args( pydantic_settings\sources.py:1328: in _add_parser_args self._add_parser_args( pydantic_settings\sources.py:1255: in _add_parser_args for field_name, resolved_name, field_info in self._sort_arg_fields(model): pydantic_settings\sources.py:1151: in _sort_arg_fields fields = model.__pydantic_fields__ if is_pydantic_dataclass(model) else model.model_fields _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self =maybe related to #303