Decorator that sets all `SQLModel` fields to `Optional`

Tomperez98 commented 2 years ago

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the SQLModel documentation, with the integrated search.
[X] I already searched in Google "How to X in SQLModel" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to SQLModel but to Pydantic.
[X] I already checked if it is not related to SQLModel but to SQLAlchemy.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code

import sqlmodel

class HeroBase(sqlmodel.SQLModel):
    name: str = sqlmodel.Field(index=True)
    secret_name: str
    age: Optional[int] = sqlmodel.Field(default=None, index=True)

    team_id: Optional[int] = sqlmodel.Field(
        default=None, foreign_key="team.id"
    )

class HeroUpdate(sqlmodel.SQLModel):
    name: Optional[str] = None
    secret_name: Optional[str] = None
    age: Optional[int] = None
    team_id: Optional[int] = None

Description

It feels bad to define every field manually to Optional. (Also prompt to error)

Wanted Solution

It would be better to have some kind of decorator or something that allows to to this at runtime

Wanted Code

import sqlmodel

class HeroBase(sqlmodel.SQLModel):
    name: str = sqlmodel.Field(index=True)
    secret_name: str
    age: Optional[int] = sqlmodel.Field(default=None, index=True)

    team_id: Optional[int] = sqlmodel.Field(
        default=None, foreign_key="team.id"
    )

@sqlmodel.all_fields_to_optional
class HeroUpdate(HeroBase):
    pass

Alternatives

No response

Operating System

Linux

Operating System Details

No response

SQLModel Version

0.0.8

Python Version

Python 3.10.6

Additional Context

No response

RobertRosca commented 2 years ago

This usecase is pretty common across a few projects using Pydantic, and it has come up a few times, but it has been postponed until Pydantic version 2. Because of that, implementing it in SQLModel itself doesn't seem like the best approach (as this could also be requested for FastAPI, or Starlite, etc...).

Instead, it might be worth commenting your support for this on existing issues https://github.com/pydantic/pydantic/pull/3179, or requesting it as a feature for Pydantic version 2: https://github.com/pydantic/pydantic/discussions/categories/pydantic-v2

Until v2 comes out, there are a few decent solutions:

The metaclass solution from Stackoverflow posted by Drdilyor looks quite nice and should accomplish what you want:

from typing import Optional

from fastapi import FastAPI
from pydantic import BaseModel
import pydantic

app = FastAPI()

class Item(BaseModel):
    name: str
    description: str
    price: float
    tax: float

class AllOptional(pydantic.main.ModelMetaclass):
    def __new__(self, name, bases, namespaces, **kwargs):
        annotations = namespaces.get('__annotations__', {})
        for base in bases:
            annotations.update(base.__annotations__)
        for field in annotations:
            if not field.startswith('__'):
                annotations[field] = Optional[annotations[field]]
        namespaces['__annotations__'] = annotations
        return super().__new__(self, name, bases, namespaces, **kwargs)

class UpdatedItem(Item, metaclass=AllOptional):
    pass

phi-friday commented 2 years ago

This usecase is pretty common across a few projects using Pydantic, and it has come up a few times, but it has been postponed until Pydantic version 2. Because of that, implementing it in SQLModel itself doesn't seem like the best approach (as this could also be requested for FastAPI, or Starlite, etc...).

Instead, it might be worth commenting your support for this on existing issues pydantic/pydantic#3179, or requesting it as a feature for Pydantic version 2: https://github.com/pydantic/pydantic/discussions/categories/pydantic-v2

Until v2 comes out, there are a few decent solutions:

Comparison 2 fields of model between himself pydantic/pydantic#2272 (comment)

https://stackoverflow.com/questions/67699451/make-every-fields-as-optional-with-pydantic

The metaclass solution from Stackoverflow posted by Drdilyor looks quite nice and should accomplish what you want:
from typing import Optional

from fastapi import FastAPI
from pydantic import BaseModel
import pydantic

app = FastAPI()

class Item(BaseModel):
    name: str
    description: str
    price: float
    tax: float

class AllOptional(pydantic.main.ModelMetaclass):
    def __new__(self, name, bases, namespaces, **kwargs):
        annotations = namespaces.get('__annotations__', {})
        for base in bases:
            annotations.update(base.__annotations__)
        for field in annotations:
            if not field.startswith('__'):
                annotations[field] = Optional[annotations[field]]
        namespaces['__annotations__'] = annotations
        return super().__new__(self, name, bases, namespaces, **kwargs)

class UpdatedItem(Item, metaclass=AllOptional):
    pass

@RobertRosca This solution has no problem at runtime, but unfortunately pylance does not recognize it.

RobertRosca commented 2 years ago

I'm not familiar with how pylance works, but it's only a static type checker which doesn't execute any code during its analysis, so it won't be aware of changes made to the annotations which happen at runtime. I don't know if there's an elegant way around this problem.

The easiest option would be to stick with defining an additional class with everything set to Optional as before, alternatively you can try generating type stub files automatically which might pick up the runtime type changes, but I haven't tested this out or used automatic stub generators so it may not work.

Tomperez98 commented 2 years ago

Closing it. Not SQLModel problem

tiangolo commented 1 year ago

Thanks for the discussion and for coming back to close it! ☕

For completeness, yep, the thing is that it's actually part of how the Python language works. This would probably be a feature request to Python itself. A type defined as a string is indeed a different type then the union of a string and None. Those two are different in several ways.

And supporting something like this would mean that your code, your editor, tools, would be thinking that your code means something, checking for some specific errors, giving you autocompletion for some specific things, and it would all be wrong. The intention of SQLModel is to make it easy to get all those features correctly, and to be able to be sure that your code is correct, at least in terms of that.

The other thing is that JSON Schema and OpenAPI would also be broken, and if you have automatically generated clients or developers using your OpenAPI (e.g. Swagger in /docs) they would all have incorrect information, invalid types. Which is even worse than no types.

fastapi / sqlmodel