Could not find any class in the compiled python file

[openedhardware]

Hi, guys.

I have just followed the tutorial (https://developers.google.com/protocol-buffers/docs/pythontutorial) to compile my .proto file and generate python file.

In this tutorial, they say generated python file is something like this:

class Person(message.Message):
  __metaclass__ = reflection.GeneratedProtocolMessageType

 class PhoneNumber(message.Message):
    __metaclass__ = reflection.GeneratedProtocolMessageType
      DESCRIPTOR = _PERSON_PHONENUMBER
     DESCRIPTOR = _PERSON

 class AddressBook(message.Message):
   __metaclass__ = reflection.GeneratedProtocolMessageType
   DESCRIPTOR = _ADDRESSBOOK

But unfortunately generated python file is quiet different with this..

Seems like there is an update for protobuf and it has been changed.

And they say I can use ParseFromString(data) function to parse a message from the given string.

But generated class has no function at all...

Is there any document?

[mjmdavis]

@openedhardware there is some documentation here: https://developers.google.com/protocol-buffers/docs/pythontutorial

It seems to me that the classes are built at runtime and the generated python is a bunch of descriptors.

Is there some code completion engine that will play nice with these dynamically generated classes? It's also a shame not to be able to use the type hints in python3.

[mjmdavis]

Also curious to understand the reasoning behind the approach used. Why not just generate classes directly?

[xfxyjwf]

Python generated API is documented here: https://developers.google.com/protocol-buffers/docs/reference/python-generated

Though I don't know why it's that way from the beginning, I kind of think that's the right way to go with for dynamic typed languages. The benefits: 1. minimum generated code size which is important in a system where tens of thousands of .proto files are used. In Google we have some really big binaries half of which is just the proto generated code and we have to spend time on shrinking the size or otherwise some binaries can not be compiled any more... 2. the interface between generated code and the protobuf runtime is minimum and that makes it far more easier to remain binary backwards compatible when we make changes to protobuf (i.e,. if the generated code doesn't change at all, a program compiled using an older version of protobuf can switch to depend on a newer version protobuf without needing any change).

[mjmdavis]

@xfxyjwf Interesting to hear your thoughts. The objective-c implementation seems to do something similar but has the advantage of a header file to provide an interface. Maybe there's some way to include an interface or to fake the class and replace it at runtime in python.

To me, one of the huge advantages of using protobuf is having types and field names available for static checks in the editor. Python3 type hints and autocomplete would really help productivity and reduce errors. Is there some way to achieve this that I'm missing?

[openedhardware]

Hi, guys. Sorry for delay.

My point is just compile this .proto file.

 syntax = "proto2";

package PKG_TripUpdate;

message TripTable{
message Header {
    optional string gtfs_realtime_version = 1;
    optional string incrementality = 2;
    optional int32 timestamp = 3;
}
optional Header header = 1;

message Entity {
    required string id = 1;
    message TripUpdate {
        message Trip {
            required string trip_id = 1;
            optional string start_time = 2;
            optional string start_date = 3;
            optional string schedule_relationship = 4;
            optional string route_id = 5;
        }
        required Trip trip = 1;

        message StopTimeUpdate {
            optional int32 stop_sequence = 1;

            message Arrival {
                optional int32 delay = 1;
                optional int32 time = 2;
            }
            optional Arrival arrival = 2;

            message Departure {
                optional int32 delay = 1;
                optional int32 time = 2;
            }
            optional Departure departure = 3;

            optional string stop_id = 4;

            optional string schedule_relationship = 5;

        }
        repeated StopTimeUpdate stop_time_update = 2;

        message Vehicle {
            optional string id = 1;
            optional string no_1999 = 2;
        }
        optional Vehicle vehicle = 3;

        optional int32 no_4 = 4;

    }
    required TripUpdate tripupdate = 2;

}
repeated Entity entity = 2;
}    

After compiling, I am not sure how to use compiled class. There is no functions, callable members at all.

[lyschoening]

@openedhardware You should see something the code below. That is actually your class.

TripTable = _reflection.GeneratedProtocolMessageType('TripTable', (_message.Message,), dict(
...

I would just like to weigh in and say that the Python code generation is absolutely horrible. No IDE will be able to code complete this mess. Judging from the sum of issues with Protocol Buffers and gRPC it doesn't seem as if there is any real interest in decent Python support.

[pviotti]

This is a major inconvenience for developers.
Is there a way to have static checks and autocomplete on generated code with any version of Python at all?

related: #2638

[RawandKurdy]

Python support sucks really, Javascript is also the same, I guess the only language that is decent to use with gRPC is Go (Speaking of the Server Side perspective)

[d-j-kendall]

So I essentially have a bunch of generated code via reflection, but I can't actually see which methods and fields I have access to?

[Manu10744]

This seems to work better (havent tried out myself yet): https://github.com/danielgtaylor/python-betterproto

The default implementation by google is absolute bullshit as of now. And, considering the date this issue was created, apparently no one at google is interested in changing that. 👎

[fowles]

Support for enerating type stubs via .pyi files has been available for a few months. The actual request here of generating more naive python code has incredibly detrimental affects on performance and will not be done.

https://github.com/protocolbuffers/protobuf/issues/2638