nipunn1313 / mypy-protobuf

open source tools to generate mypy stubs from protobufs
Apache License 2.0
654 stars 80 forks source link

Unexpected type arguments in generated code for synchronous `UnaryUnaryMultiCallable` #629

Open ArthurBook opened 3 months ago

ArthurBook commented 3 months ago

Hi, thanks for the work on this project. It really improves the development workflow to have the proto and grpc interfaces recognizable by the language server (vscode pyright in my case).

The confusion / bug

When inspecting the generated .pyi file, I notice that the method type referenced in the client stub: grpc.UnaryUnaryMultiCallable is hinted with two type parameters:

class Service1Stub:
    def __init__(self, channel: typing.Union[grpc.Channel, grpc.aio.Channel]) -> None: ...
    GetUser: grpc.UnaryUnaryMultiCallable[
        service_pb2.GetUserRequest,
        service_pb2.GetUserResponse,
    ]

However, in the source implementation there are no generic typevars bound to the grpc.UnaryUnaryMultiCallable.

It looks like there are typevars bound to the async version of the async version of unary-unary RPC that can be found here.

Let me know if something might be misconfigured on my side -- elsewise, more than happy to open a PR that suggests adding in the RequestType and ResponseType generics to the synchronous unary-unary in the grpc repo!

Background / replication

protobuf : 5.27.3   
grpcio-tools : 1.65.4                           
mypy-protobuf : 3.6.0                                        

Simplen sample client stub .proto file:

syntax = "proto3";

package common;

message User {
  int32 id = 1;
  string name = 2;
}

service Service1 {
  rpc GetUser(GetUserRequest) returns (GetUserResponse);
}

message GetUserRequest {
  int32 user_id = 1;
}

message GetUserResponse {
  User user = 1;
}

By running:

python -m grpc_tools.protoc \
                -Iproto \
                --python_out=client/generated \
                --mypy_out=client/generated \
                --grpc_python_out=client/generated \
                --mypy_grpc_out=client/generated \
                proto/*.proto

...It produces the following pb2_grpc_.pyi stub file:

"""
@generated by mypy-protobuf.  Do not edit manually!
isort:skip_file
"""

import abc
import collections.abc
import grpc
import grpc.aio
import service_pb2
import typing

_T = typing.TypeVar("_T")

class _MaybeAsyncIterator(collections.abc.AsyncIterator[_T], collections.abc.Iterator[_T], metaclass=abc.ABCMeta): ...

class _ServicerContext(grpc.ServicerContext, grpc.aio.ServicerContext):  # type: ignore[misc, type-arg]
    ...

class Service1Stub:
    def __init__(self, channel: typing.Union[grpc.Channel, grpc.aio.Channel]) -> None: ...
    GetUser: grpc.UnaryUnaryMultiCallable[
        service_pb2.GetUserRequest,
        service_pb2.GetUserResponse,
    ]

class Service1AsyncStub:
    GetUser: grpc.aio.UnaryUnaryMultiCallable[
        service_pb2.GetUserRequest,
        service_pb2.GetUserResponse,
    ]

class Service1Servicer(metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def GetUser(
        self,
        request: service_pb2.GetUserRequest,
        context: _ServicerContext,
    ) -> typing.Union[service_pb2.GetUserResponse, collections.abc.Awaitable[service_pb2.GetUserResponse]]: ...

def add_Service1Servicer_to_server(servicer: Service1Servicer, server: typing.Union[grpc.Server, grpc.aio.Server]) -> None: ...
nipunn1313 commented 3 months ago

Yeah that's interesting. It's a little surprising that only the async stub has @Evgenus worked on this originally and may have some insight.

I am curious what concrete issue you're running into as a result of this. Is there a typecheck issue that comes up?

mypy-protobuf has tests https://github.com/nipunn1313/mypy-protobuf/blob/main/proto/testproto/grpc/import.proto which generates this stub https://github.com/nipunn1313/mypy-protobuf/blob/main/test/generated/testproto/grpc/import_pb2_grpc.pyi Which typechecks (our CI checks it).

ArthurBook commented 3 months ago

It's a minor issue, but since the UnaryUnaryMultiCallable.__call__ doesn't have types, the return type of the generated RPC request method on the client will be unknown for type checkers.

Example

import grpc
from shared.generated_code import service_pb2, service_pb2_grpc

def run() -> None:
    channel = grpc.insecure_channel("localhost:50051")
    stub = service_pb2_grpc.Service1Stub(channel)
    request = service_pb2.GetUserRequest(user_id=123)
    response = stub.GetUser(request) # (variable) response: Unknown

if __name__ == "__main__":
    run()