MichaelLan commented 2 years ago

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the SQLModel documentation, with the integrated search.
[X] I already searched in Google "How to X in SQLModel" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to SQLModel but to Pydantic.
[X] I already checked if it is not related to SQLModel but to SQLAlchemy.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code

# database.py

from sqlmodel import Session, create_engine
from core.settings import settings

engine = create_engine(settings.SQLALCHEMY_DATABASE_URL, echo=True)

def get_session():
    with Session(engine) as session:
        yield session

# models.py

from typing import Optional

from sqlmodel import Field, SQLModel

class Companies(SQLModel, table=True):
    avatar: Optional[str] = None
    business_name: Optional[str] = None
    country_id: int = Field(index=True)
    created: str
    crm_id: Optional[str] = None
    id: Optional[str] = Field(default=None, primary_key=True)
    is_active: bool
    is_removed: bool
    metadata_: Optional[str] = Field(default=None, alias="metadata")  # Error: Companies.metadata_ does not exist

    # before
    metadata: Optional[str] = Field(default=None)  # Error: NameError

Description

I created a model but this have a field named metadata, is an existing database, so I can't change the fields names, since the SQLModel class also has a field called metadata, when I run the app I get the following error NameError: Field name "metadata" shadows a BaseModel attribute; use a different field name with "alias='metadata'".

I used the Field class of sqlmodel and assigned the alias but it does not work. metadata_: Optional[str] = Field(default=None, alias="metadata") With this the error is that Table.metadata_ field does not exist

Is there any way to create an tabla with the metadata field without giving problems with SQLModel?

Operating System

Linux

Operating System Details

No response

SQLModel Version

0.0.6

Python Version

Python 3.9.5

Additional Context

No response

MichaelLan commented 2 years ago

I have found the solution using the sacolumn argument for the Field class, but the alias argument in the SQLModel class should work. example code. `metadata: Optional[str] = Field(sa_column=Column("metadata", String, default=None))`

tcztzy commented 1 month ago

Would this problem be fixed? Do we have any walkaround for metadata's validation/serialization currently?

tcztzy commented 1 month ago

Would this problem be fixed? Do we have any walkaround for metadata's validation/serialization currently?

Some research

from pydantic import BaseModel
from pydantic import Field as PydanticField
from sqlmodel import Field, SQLModel

class PydanticModelWithPydanticField(BaseModel):
    metadata_: str = PydanticField(alias="metadata")

class PydanticModelWithSQLModelField(BaseModel):
    metadata_: str = Field(alias="metadata")

class SQLModelWithPydanticField(SQLModel):
    metadata_: str = PydanticField(alias="metadata")

class SQLModelWithSQLModelField(SQLModel):
    metadata_: str = Field(alias="metadata")

if __name__ == "__main__":
    try:
        sql_model_no_suffix = SQLModelWithSQLModelField(metadata="metadata")
        print(f'{sql_model_no_suffix.model_dump(by_alias=False)=}')
        print(f'{sql_model_no_suffix.model_dump(by_alias=True)=}')
    except Exception as e:
        print(f'{e=}')

    sql_model_with_suffix = SQLModelWithSQLModelField(metadata_="metadata")
    print(f'{sql_model_with_suffix.model_dump(by_alias=False)=}')
    print(f'{sql_model_with_suffix.model_dump(by_alias=True)=}')

    sql_model_with_pydantic_field = SQLModelWithPydanticField(metadata="metadata")
    print(f'{sql_model_with_pydantic_field.model_dump(by_alias=False)=}')
    print(f'{sql_model_with_pydantic_field.model_dump(by_alias=True)=}')

    try:
        sql_model_with_pydantic_field_with_suffix = SQLModelWithPydanticField(metadata_="metadata")
        print(f'{sql_model_with_pydantic_field_with_suffix.model_dump(by_alias=False)=}')
        print(f'{sql_model_with_pydantic_field_with_suffix.model_dump(by_alias=True)=}')
    except Exception as e:
        print(f'{e=}')

    pydantic_model_with_pydantic_field = PydanticModelWithPydanticField(metadata="metadata")
    print(f'{pydantic_model_with_pydantic_field.model_dump(by_alias=False)=}')
    print(f'{pydantic_model_with_pydantic_field.model_dump(by_alias=True)=}')

    try:
        pydantic_model_with_pydantic_field_with_suffix = PydanticModelWithPydanticField(metadata_="metadata")
        print(f'{pydantic_model_with_pydantic_field_with_suffix.model_dump(by_alias=False)=}')
        print(f'{pydantic_model_with_pydantic_field_with_suffix.model_dump(by_alias=True)=}')
    except Exception as e:
        print(f'{e=}')

    try:
        pydantic_model_no_suffix = PydanticModelWithSQLModelField(metadata="metadata")
        print(f'{pydantic_model_no_suffix.model_dump(by_alias=False)=}')
        print(f'{pydantic_model_no_suffix.model_dump(by_alias=True)=}')
    except Exception as e:
        print(f'{e=}')

    pydantic_model_with_sql_model_field_with_suffix = PydanticModelWithSQLModelField(metadata_="metadata")
    print(f'{pydantic_model_with_sql_model_field_with_suffix.model_dump(by_alias=False)=}')
    print(f'{pydantic_model_with_sql_model_field_with_suffix.model_dump(by_alias=True)=}')

output of previous code

e=1 validation error for SQLModelWithSQLModelField
metadata_
  Field required [type=missing, input_value={'metadata': 'metadata'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.8/v/missing
sql_model_with_suffix.model_dump(by_alias=False)={'metadata_': 'metadata'}
sql_model_with_suffix.model_dump(by_alias=True)={'metadata_': 'metadata'}
sql_model_with_pydantic_field.model_dump(by_alias=False)={'metadata_': 'metadata'}
sql_model_with_pydantic_field.model_dump(by_alias=True)={'metadata': 'metadata'}
e=1 validation error for SQLModelWithPydanticField
metadata
  Field required [type=missing, input_value={'metadata_': 'metadata'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.8/v/missing
pydantic_model_with_pydantic_field.model_dump(by_alias=False)={'metadata_': 'metadata'}
pydantic_model_with_pydantic_field.model_dump(by_alias=True)={'metadata': 'metadata'}
e=1 validation error for PydanticModelWithPydanticField
metadata
  Field required [type=missing, input_value={'metadata_': 'metadata'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.8/v/missing
e=1 validation error for PydanticModelWithSQLModelField
metadata_
  Field required [type=missing, input_value={'metadata': 'metadata'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.8/v/missing
pydantic_model_with_sql_model_field_with_suffix.model_dump(by_alias=False)={'metadata_': 'metadata'}
pydantic_model_with_sql_model_field_with_suffix.model_dump(by_alias=True)={'metadata_': 'metadata'}

Current walk around

I am inspired by this, write a quick and dirty walk around

from contextlib import asynccontextmanager

from fastoai import FastOAI
from fastoai.routers.ollama import router
from fastoai.settings import get_settings
from starlette.datastructures import MutableHeaders
from starlette.types import ASGIApp, Message, Receive, Scope, Send

class MetaDataAdderMiddleware:
    application_generic_urls = ['/openapi.json', '/docs', '/docs/oauth2-redirect', '/redoc']

    def __init__(
            self, 
            app: ASGIApp
    ) -> None:
        self.app = app

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        if scope["type"] == "http" and not any([scope["path"].startswith(endpoint) for endpoint in MetaDataAdderMiddleware.application_generic_urls]):
            responder = MetaDataAdderMiddlewareResponder(self.app)
            await responder(scope, receive, send)
            return
        await self.app(scope, receive, send)

class MetaDataAdderMiddlewareResponder:

    def __init__(
            self,
            app: ASGIApp,
    ) -> None:
        """
        """
        self.app = app
        self.initial_message: Message = {}

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        self.send = send
        await self.app(scope, receive, self.send_with_meta_response)

    async def send_with_meta_response(self, message: Message):

        message_type = message["type"]
        if message_type == "http.response.start":
            # Don't send the initial message until we've determined how to
            # modify the outgoing headers correctly.
            self.initial_message = message

        elif message_type == "http.response.body":
            data_to_be_sent_to_user = message["body"].replace(b'"metadata_"', b'"metadata"')

            headers = MutableHeaders(raw=self.initial_message["headers"])
            headers["Content-Length"] = str(len(data_to_be_sent_to_user))
            message["body"] = data_to_be_sent_to_user

            await self.send(self.initial_message)
            await self.send(message)

settings = get_settings()

@asynccontextmanager
async def lifespan():
    yield
    settings.session.close()

app = FastOAI(root_path="/v1", lifespan=lifespan)

app.add_middleware(MetaDataAdderMiddleware)

# Order matters, you should firstly define your own entrypoint functions, then
# include the existing router in order to override the original process with
# yours
app.include_router(router)

conclusion

This problem seems come from SQLModel's Field

fastapi / sqlmodel

Problem with the metadata field (NameError: Field name "metadata" shadows a BaseModel attribute; use a different field name with "alias='metadata'".) #290