Bug: DTOData class not behaving as expected and PydanticDTO class error validation

[HorusTheSonOfOsiris]

Description

The DTOData class works as expected when there isn't a nested model. Based on the documentation, the idea of DTOData class is to delay the validation until create_instance method is called.

This is working as expected for the first level model but calls Pydantic validation on nested models.

Second issue: I am facing an issue when overriding the ValidationException class. I am using an extra field in the error response which is not serialised properly. An internal error is raised:

Traceback (most recent call last):
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/contrib/pydantic/pydantic_dto_factory.py", line 101, in decode_bytes
    return super().decode_bytes(value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/dto/base_dto.py", line 97, in decode_bytes
    return backend.populate_data_from_raw(value, self.asgi_connection)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/dto/_codegen_backend.py", line 143, in populate_data_from_raw
    data_as_builtins=self._transfer_to_dict(self.parse_raw(raw, asgi_connection)),
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 22, in func
  File "<string>", line 22, in <genexpr>
  File "<string>", line 25, in func
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/pydantic/main.py", line 176, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for B
nested_c
  List should have at most 1 item after validation, not 2 [type=too_long, input_value=[C(i_am_c=None), C(i_am_c=None)], input_type=list]
    For further information visit https://errors.pydantic.dev/2.7/v/too_long

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/middleware/_internal/exceptions/middleware.py", line 158, in __call__
    await self.app(scope, receive, capture_response_started)
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/_asgi/asgi_router.py", line 99, in __call__
    await asgi_app(scope, receive, send)
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/routes/http.py", line 80, in handle
    response = await self._get_response_for_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/routes/http.py", line 132, in _get_response_for_request
    return await self._call_handler_function(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/routes/http.py", line 152, in _call_handler_function
    response_data, cleanup_group = await self._get_response_data(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/routes/http.py", line 176, in _get_response_data
    data = await kwargs["data"]
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/_kwargs/extractors.py", line 501, in dto_extractor
    return data_dto(connection).decode_bytes(body)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/contrib/pydantic/pydantic_dto_factory.py", line 103, in decode_bytes
    raise ValidationException(extra=convert_validation_error(ex)) from ex
litestar.exceptions.http_exceptions.ValidationException: 400: Bad Request

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/serialization/msgspec_hooks.py", line 162, in encode_json
    return msgspec.json.encode(value, enc_hook=serializer) if serializer else _msgspec_json_encoder.encode(value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/serialization/msgspec_hooks.py", line 92, in default_serializer
    raise TypeError(f"Unsupported type: {type(value)!r}")
TypeError: Unsupported type: <class 'app.C'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 399, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/app.py", line 591, in __call__
    await self.asgi_handler(scope, receive, self._wrap_send(send=send, scope=scope))  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/middleware/_internal/exceptions/middleware.py", line 175, in __call__
    await self.handle_request_exception(
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/middleware/_internal/exceptions/middleware.py", line 205, in handle_request_exception
    await response.to_asgi_response(app=None, request=request)(scope=scope, receive=receive, send=send)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/response/base.py", line 451, in to_asgi_response
    body=self.render(self.content, media_type, get_serializer(type_encoders)),
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/response/base.py", line 392, in render
    return encode_json(content, enc_hook)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/redacted/Library/Caches/pypoetry/virtualenvs/redacted/lib/python3.12/site-packages/litestar/serialization/msgspec_hooks.py", line 164, in encode_json
    raise SerializationException(str(msgspec_error)) from msgspec_error
litestar.exceptions.base_exceptions.SerializationException: Unsupported type: <class 'app.C'>

URL to code causing the issue

No response

MCVE

from typing import Generic, TypeVar

from litestar import Controller, Litestar, Request, Response, post
from litestar.contrib.pydantic import PydanticDTO
from litestar.dto import DTOConfig, DTOData
from litestar.exceptions import ValidationException
from pydantic import BaseModel, Field

IdVar = TypeVar("IdVar")

def router_handler_exception_handler(
    request: Request, exc: ValidationException
) -> Response:
    return Response(
        content={
            "error": "validation error",
            "path": request.url.path,
            "extra": exc.extra,
        },
        status_code=400,
    )

class BaseA(BaseModel, Generic[IdVar]):
    id_var: IdVar = None

class C(BaseModel):
    i_am_c: str | None = None

class B(BaseA[int], Generic[IdVar]):
    i_am_b: str | None
    nested_c: list[C] = Field(default=[], max_length=1)

class A(BaseA[str], Generic[IdVar]):
    i_am_a: str | None
    nested_b: list[B] = Field(default=[], max_length=1)

class WriteADto(PydanticDTO[A]):
    config = DTOConfig(
        rename_strategy="camel",
        max_nested_depth=3,
        exclude={"id_var", "nested_b.0.id_var"},
    )

class Test(Controller):
    @post(path="/test", dto=WriteADto)
    async def test(self, data: DTOData[A]) -> None:
        print(data.create_instance())

app = Litestar(
    route_handlers=[Test],
    exception_handlers={ValidationException: router_handler_exception_handler},
)

Steps to reproduce

For Issue: 1

1. Call the /test endpoint with following data:

   {
   "iAmA": null,
   "nestedB": [
   {
     "iAmB": null,
     "nestedC": [
       {
         "iAmC": null
       }
     ]
   },
   {
     "iAmB": null,
     "nestedC": [
       {
         "iAmC": null
       }
     ]
   }
   ]
   }

2. Works without raising an error.
3. Comment out exception_handlers from app.
4. Call /test endpoint with following data:

   {
   "iAmA": null,
   "nestedB": [
   {
     "iAmB": null,
     "nestedC": [
       {
         "iAmC": null
       },
       {
         "iAmC": null
       }
     ]
   }
   ]
   }

5. Raises an internal server error and ValidationException.

For Issue 2:

1. Enable ValidationException in the app.

2. Call /test endpoint with following data:

   {
   "iAmA": null,
   "nestedB": [
   {
     "iAmB": null,
     "nestedC": [
       {
         "iAmC": null
       },
       {
         "iAmC": null
       }
     ]
   }
   
   ]
   }

3. Raises an internal server error and ValidationException and SerializationException.

Screenshots

"![SCREENSHOT_DESCRIPTION](SCREENSHOT_LINK.png)"

Logs

No response

Litestar Version

2.9.0final0

Platform

• [ ] Linux
• [X] Mac
• [ ] Windows
• [ ] Other (Please specify in the description above)

---

[!NOTE]
While we are open for sponsoring on GitHub Sponsors and OpenCollective, we also utilize Polar.sh to engage in pledge-based sponsorship.

Check out all issues funded or available for funding on our Polar.sh dashboard

• If you would like to see an issue prioritized, make a pledge towards it!
• We receive the pledge once the issue is completed & verified
• This, along with engagement in the community, helps us know which features are a priority to our users.

[HorusTheSonOfOsiris]

I am raising two issues together because they are somewhat interrelated. For Issue 2 where SerializationException is encountered. I believe that can be resolved by changing the convert_validation_error method

def convert_validation_error(validation_error: ValidationErrorV1 | ValidationErrorV2) -> list[dict[str, Any]]:
    error_list = validation_error.errors()
    for error in error_list:
        if isinstance(exception := error.get("ctx", {}).get("error"), Exception):
            error["ctx"]["error"] = type(exception).__name__
    return error_list  # type: ignore[return-value]

We can change to following code:

def convert_validation_error(validation_error: ValidationErrorV1 | ValidationErrorV2) -> list[dict[str, Any]]:
    error_list = json.loads(validation_error.json())
    for error in error_list:
        if isinstance(exception := error.get("ctx", {}).get("error"), Exception):
            error["ctx"]["error"] = type(exception).__name__
    return error_list  # type: ignore[return-value]

This way Pydantic handle the nested model conversion.