Importing well known types

Luminaar commented 4 years ago

Hello,

I'm not sure how to work with well known types.

Having a proto file like this:

syntax = "proto3";      

package simple;      

import "google/protobuf/empty.proto";      

service SimpleSender {      
    rpc Send(SendParams) returns (google.protobuf.Empty) {}      
}      

message SendParams {      
    string body = 1;      
}

I get output like this:

# Generated by the protocol buffer compiler.  DO NOT EDIT!      
# sources: Simple.proto      
# plugin: python-betterproto      
from dataclasses import dataclass      

import betterproto      
import grpclib          

from .google import protobuf      

@dataclass      
class SendParams(betterproto.Message):      
    body: str = betterproto.string_field(1)      

class SimpleSenderStub(betterproto.ServiceStub):      
    async def send(self, *, body: str = "") -> protobuf.Empty:      
        request = SendParams()      
        request.body = body         

        return await self._unary_unary(      
            "/simple.SimpleSender/Send", request, protobuf.Empty,      
        )

There is ofcourse no google package. I tried to generate code from proto files included in grpc_tools but I'm not able to get any python code.

What is the best way to work with well known types?

danielgtaylor commented 4 years ago

It looks like the google.protobuf.Emtpy type is not currently supported. Someone will need to implement it in python-betterproto.

As a workaround you can define your own empty message until this is done.

JoaoVasques commented 4 years ago

Howdy people!

The same issue happens when I tried to use Struct.

from unbabelevents.chat_api import *
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-2-4950c13b1c3b> in <module>
----> 1 from unbabelevents.chat_api import *
~/chat_api.py in <module>
      9
     10 from . import commons
---> 11 from .google import protobuf
     12
     13

ModuleNotFoundError: No module named 'unbabelevents.google'

The proto file has the following message

syntax = "proto3";

import "google/protobuf/struct.proto";

message Nugget {
    repeated google.protobuf.Struct alerts = 1;
}

And the generated python class is


import betterproto

from . import commons
from .google import protobuf

@dataclass
class Nugget(betterproto.Message):
    qe_alerts: List[protobuf.Struct] = betterproto.message_field(1)

devova commented 4 years ago

It looks like the google.protobuf.Emtpy type is not currently supported. Someone will need to implement it in python-betterproto.

There is a google.protobuf.empty_pb2.Empty

adisunw commented 4 years ago

@devova yes but there are no generated dataclasses.

I made this module as a bandaid.


#  proto_types.py
from dataclasses import dataclass
import betterproto
from google.protobuf import struct_pb2

@dataclass
class Struct(betterproto.Message):
    def parse(self: betterproto.T, data: bytes) -> betterproto.T:
        decoded = struct_pb2.Struct.FromString(data)
        self.__dict__.update({item: value for item, value in decoded.items()})
        return self

    def __repr__(self):
        return repr(self.__dict__)

@dataclass
class Empty(betterproto.Message):
    def parse(self: betterproto.T, data: bytes) -> betterproto.T:
        return self

And then a post-generating find and replace.

with open(file_path, 'r') as f:
    lines = f.readlines()
with open(file_path, 'w') as f:
    for line in lines:
        line = line.replace("from .google import protobuf", "from wherever_proto_types_exists import proto_types")
        line = line.replace('protobuf.Empty', 'proto_types.Empty')
        line = line.replace('protobuf.Struct', 'proto_types.Struct')
        f.write(line)

where file_path is the path to the generated grpc python module

kalvdans commented 4 years ago

Fixed by #78?

kalzoo commented 3 years ago

Not Struct, unfortunately. If Struct is the only WKT used in your protobuf (as it is for me), you'll still have this dangling/invalid import. @adisunw 's patch fix at least unblocks the build, which is great.

Even easier than the find-replace that @adisunw provides above, if you just put that file in google/protobuf.py relative to the imported files you're in business.