search

litestar-org / litestar

Production-ready, Light, Flexible and Extensible ASGI API framework | Effortlessly Build Performant APIs
https://litestar.dev/
MIT License
5.51k stars 376 forks source link

DTO : TypeError: 'NoneType' object is not iterable #1761

Closed v3ss0n closed 1 year ago

v3ss0n commented 1 year ago

Description

After updating to 76696a2ae243ed05c4361e9f4dda3e06d0859e16 DTOs i have , having this error .

TypeError: 'NoneType' object is not iterable

URL to code causing the issue

No response

MCVE

https://github.com/v3ss0n/starlite-pg-redis-docker/tree/repro-dto-nontype

Steps to reproduce

  1. Make a request to Projects API 
    curl -X 'GET' \
'http://localhost:8000/v1/api/projects?page=1&page-size=100' \
-H 'accept: application/json'
  2. See error

Screenshots

"![SCREENSHOT_DESCRIPTION](SCREENSHOT_LINK.png)"

Logs

Traceback (most recent call last):
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/middleware/exceptions/middleware.py", line 150, in __call__
    await self.app(scope, receive, send)
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 77, in handle
    response = await self._get_response_for_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 129, in _get_response_for_request
    response = await self._call_handler_function(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 162, in _call_handler_function
    response: ASGIApp = await route_handler.to_response(app=scope["app"], data=response_data, request=request)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/handlers/http_handlers/base.py", line 469, in to_response
    return await response_handler(app=app, data=data, request=request, return_dto=self.resolve_return_dto())  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/handlers/http_handlers/_utils.py", line 99, in handler
    data = return_dto(ctx).data_to_encodable_type(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/abc.py", line 103, in data_to_encodable_type
    return backend.encode_data(data, self.connection_context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/abc.py", line 286, in encode_data
    return transfer_data(
           ^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/utils.py", line 140, in transfer_data
    return type(source_data)(
           ^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/utils.py", line 141, in <genexpr>
    transfer_data(destination_type, item, field_definitions, dto_for)  # type:ignore[call-arg]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/utils.py", line 144, in transfer_data
    return transfer_instance_data(destination_type, source_data, field_definitions, dto_for)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/utils.py", line 196, in transfer_instance_data
    unstructured_data[destination_name] = transfer_type_data(
                                          ^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/utils.py", line 240, in transfer_type_data
    return transfer_type.parsed_type.origin(source_value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable

Litestar Version

76696a2ae243ed05c4361e9f4dda3e06d0859e16

Platform

v3ss0n commented 1 year ago

I couldn't do it in minimal way so i pushed a repo , based on full-stack . Also the error has been changed to TypeError: Sequence() takes no arguments after updating to latest commit 

Traceback (most recent call last):
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/middleware/exceptions/middleware.py", line 150, in __call__
    await self.app(scope, receive, send)
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 77, in handle
    response = await self._get_response_for_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 129, in _get_response_for_request
    response = await self._call_handler_function(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 162, in _call_handler_function
    response: ASGIApp = await route_handler.to_response(app=scope["app"], data=response_data, request=request)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/handlers/http_handlers/base.py", line 469, in to_response
    return await response_handler(app=app, data=data, request=request, return_dto=self.resolve_return_dto())  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/handlers/http_handlers/_utils.py", line 99, in handler
    data = return_dto(ctx).data_to_encodable_type(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/abc.py", line 103, in data_to_encodable_type
    return backend.encode_data(data, self.connection_context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/abc.py", line 300, in encode_data
    return transfer_data(
           ^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/utils.py", line 148, in transfer_data
    return origin(  # type:ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Sequence() takes no arguments

https://github.com/v3ss0n/litestar-fullstack-error-reproduce

v3ss0n commented 1 year ago

better MVCE added : https://github.com/v3ss0n/starlite-pg-redis-docker/tree/repro-dto-nontype

v3ss0n commented 1 year ago

it comes from this line (which was totally fine before update) async def filter(self, service: "Service", filters: list["FilterTypes"] = validation_skip) -> Sequence[Model]: after changing Sequence[Model] to list[Model]` - problem solved

But Sequence is fine before the update.

EDIT: MyPy don't like list[Model]

src/app/domain/projects/controllers.py:43: error: Incompatible return value type (got "Sequence[Project]", expected "List[Project]")  [return-value]
src/app/domain/backlogs/controllers.py:42: error: Incompatible return value type (got "Sequence[Backlog]", expected "List[Backlog]")  [return-value]
Found 2 errors in 2 files (checked 2 source files)
peterschutt commented 1 year ago

I couldn't do it in minimal way so i pushed a repo , based on full-stack .

Also the error has been changed to TypeError: Sequence() takes no arguments after updating to latest commit 


Traceback (most recent call last):

  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/middleware/exceptions/middleware.py", line 150, in __call__

    await self.app(scope, receive, send)

  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 77, in handle

    response = await self._get_response_for_request(

               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 129, in _get_response_for_request

    response = await self._call_handler_function(

               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 162, in _call_handler_function

    response: ASGIApp = await route_handler.to_response(app=scope["app"], data=response_data, request=request)

                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/handlers/http_handlers/base.py", line 469, in to_response

    return await response_handler(app=app, data=data, request=request, return_dto=self.resolve_return_dto())  # type: ignore

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/handlers/http_handlers/_utils.py", line 99, in handler

    data = return_dto(ctx).data_to_encodable_type(data)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/abc.py", line 103, in data_to_encodable_type

    return backend.encode_data(data, self.connection_context)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/abc.py", line 300, in encode_data

    return transfer_data(

           ^^^^^^^^^^^^^^

  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/utils.py", line 148, in transfer_data

    return origin(  # type:ignore[no-any-return]

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

TypeError: Sequence() takes no arguments

https://github.com/v3ss0n/litestar-fullstack-error-reproduce

This isn't the same error as the OP though? Sure it is the same issue?

peterschutt commented 1 year ago

I couldn't do it in minimal way so i pushed a repo , based on full-stack . Also the error has been changed to TypeError: Sequence() takes no arguments after updating to latest commit

Traceback (most recent call last):
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/middleware/exceptions/middleware.py", line 150, in __call__
    await self.app(scope, receive, send)
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 77, in handle
    response = await self._get_response_for_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 129, in _get_response_for_request
    response = await self._call_handler_function(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/routes/http.py", line 162, in _call_handler_function
    response: ASGIApp = await route_handler.to_response(app=scope["app"], data=response_data, request=request)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/handlers/http_handlers/base.py", line 469, in to_response
    return await response_handler(app=app, data=data, request=request, return_dto=self.resolve_return_dto())  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/handlers/http_handlers/_utils.py", line 99, in handler
    data = return_dto(ctx).data_to_encodable_type(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/abc.py", line 103, in data_to_encodable_type
    return backend.encode_data(data, self.connection_context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/abc.py", line 300, in encode_data
    return transfer_data(
           ^^^^^^^^^^^^^^
  File "/workspace/app/.venv/lib/python3.11/site-packages/litestar/dto/factory/_backends/utils.py", line 148, in transfer_data
    return origin(  # type:ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Sequence() takes no arguments

https://github.com/v3ss0n/litestar-fullstack-error-reproduce

I know what the issue is here and this is fixed easy enough - but I doubt this is the same as your OP.

However, I think you should consider why the service object is annotated as if it returns generic sequences from the list() method, when it knows that it returns a concrete list, e.g. this:

    async def list(self, *filters: FilterTypes, **kwargs: Any) -> Sequence[ModelT]:
        """Wrap repository scalars operation.

        Args:
            *filters: Collection route filters.
            **kwargs: Keyword arguments for attribute based filtering.

        Returns:
            The list of instances retrieved from the repository.
        """
        return await self.repository.list(*filters, **kwargs)

If this method is typed to return list[ModelT] - which is what the repository.list() method is annotated to return, then you don't need to type the return type of your handler as Sequence[ModelT] in the first place.

Generic collections are usually good to annotate function arguments with, as when you are authoring a function, you tend to know what you are doing with the argument value within the function, and so you know if you just need any old sequence, or if you are relying on particular methods of a concrete collection type.

On the other hand, using generic types for return annotations means that downstream users of that method need to inspect the object, or cast it to something else at runtime so that they know what they are actually dealing with. E.g.,:

def returns_list() -> Sequence[int]:
    return [1, 2, 3]

l = returns_list()
l.append(4)  # mypy will complain because it thinks `l` is a generic sequence which has no `append()` method

# so user needs to do this
l = list(returns_list())
l.append(4)

This example is essentially what the service object is does in that repo. It knows that self.repository.list() returns a list, but annotates the return of its own list() method as Sequence, which loses info for downstream users.

peterschutt commented 1 year ago

Created #1763 for the Sequence issue - I minimized most of the conversation around that in this issue as it not the same issue as OP.

v3ss0n commented 1 year ago

This isn't the same error as the OP though? Sure it is the same issue?

Yeah , the first issue gone after i updated to ccc9b33831620ea20a4dffa0d0a76501f2a785bb , I will try to reproduce the OP again.

peterschutt commented 1 year ago

I'll close this for now @v3ss0n - please reopen if you can reproduce.

v3ss0n commented 1 year ago

I haven't encourter the first issue yet. so its safe to close so far. thanks a lot !