Support `extra` specification on a field-by-field basis

ArtemIsmagilov commented 1 month ago

Initial Checks

[X] I have searched Google & GitHub for similar requests and couldn't find anything
[X] I have read and followed the docs and still think this feature is missing

Description

I know that we can just ignore the additional fields for the schema. However, I want to specify certain fields that need to be ignored, for example at different resolution levels.

Example.

from pydantic import BaseModel, ConfigDict

class User(BaseModel):
    model_config = ConfigDict(skip={'age': 'ignore', 'other': 'ignore'})  

    name: str

user = User(name='John Doe', age=20, job='writer')  
print(user)
#> name='John Doe'

from pydantic import BaseModel, ConfigDict

class User(BaseModel):
    model_config = ConfigDict(skip={'age': 'allow', 'other': 'ignore'})  

    name: str

user = User(name='John Doe', age=20, job='writer')  
print(user)
#> name='John Doe' age=20

...

Affected Components

[ ] Compatibility between releases
[ ] Data validation/parsing
[ ] Data serialization - .model_dump() and .model_dump_json()
[ ] JSON Schema
[ ] Dataclasses
[ ] Model Config
[ ] Field Types - adding or changing a particular data type
[ ] Function validation decorator
[ ] Generic Models
[ ] Other Model behaviour - model_construct(), pickling, private attributes, ORM mode
[ ] Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

sydney-runkle commented 1 month ago

Hi @ArtemIsmagilov,

I could also see the API getting tricky, for example, what if someone wants to allow an extra field called other? Perhaps we could implement something like

ConfigDict(extra=Extra(forbid={...}, ignore={...}, allow={...}))

where you could set wildcard * values to specify the extra behavior for all remaining unspecified fields.

In the meantime, you can use a custom validator:

from typing import Any

from pydantic import BaseModel, ConfigDict, model_validator

class User(BaseModel):
    model_config = ConfigDict(extra='allow')  

    name: str

    @model_validator(mode='before')
    @classmethod
    def check_extras(cls, data: Any) -> Any:
        if isinstance(data, dict):
            allowed_keys = [*cls.model_fields.keys(), 'age']
            return {k: v for k, v in data.items() if k in allowed_keys}
        return data

user = User(name='John Doe', age=20, job='writer')  
print(repr(user))
#> User(name='John Doe', age=20)

ArtemIsmagilov commented 1 month ago

Great, thanks

ArtemIsmagilov commented 4 weeks ago

@sydney-runkle Are there any plans to implement this functionality?

sydney-runkle commented 3 weeks ago

@ArtemIsmagilov,

I haven't yet added it to a milestone. It'll require pretty significant changes in pydantic-core.

I'd be happy to support development on this front with PR reviews, etc if anyone is interested in taking this on, but I don't think we'll pick this up until we've addressed some other more pressing things like performance improvements, etc.

pydantic / pydantic