golang / protobuf

Go support for Google's protocol buffers
BSD 3-Clause "New" or "Revised" License
9.64k stars 1.58k forks source link

Accessing undelying Go type using `protoreflect` #1548

Closed topofstack closed 1 year ago

topofstack commented 1 year ago

I'm pretty new to protobuf reflection and the question may sound pretty dumb. Is there any other way I can get access to underlying Go type using protoreflect?

Here's small proto which includes another widely used proto:

import "google/protobuf/timestamp.proto";

message MyMessage {
    string name = 1;
    google.protobuf.Timestamp tstamp = 2;
}

Here's the code I use to parse incoming byte array on the basis of the model generated by proto-compiler

// instantiate a message generated by protoc
m := desc.MyMessage{
     Name:   "AwesomeName",
     Tstamp: timestamppb.New(time.Now()),
}
bytes := Marshal(m) // serialize message into byte array

// let the package know about the stucture of the MyMessage
msg := dynamicpb.NewMessage(m.ProtoReflect().Descriptor())
err := proto.Unmarshal(bytes, msg)
if err != nil {
    return nil, err
}
msg.Range(func(fd protoreflect.FieldDescriptor, v protoreflect.Value) bool {
        if fd.Kind() == protoreflect.MessageKind {
            if fd.Message().FullName() == "google.protobuf.Timestamp" { // have to find it by unique name :(
                timestampMsg := msg.Get(fd).Message()

                secondsDesc := timestampMsg.Descriptor().Fields().ByName("seconds")
                secondsVal := timestampMsg.Get(secondsDesc).Int()

                nanosDesc := timestampMsg.Descriptor().Fields().ByName("nanos")
                nanosVal := timestampMsg.Get(nanosDesc).Int()

                                goTime := time.Unix(secondsVal, nanosVal) // <<-- finally got Go Time object
            }
        }
        return true
    })

Is there any smarter way to implement it?

puellanivis commented 1 year ago

I’m not sure why you would want to use dynamicpb when you have the actual message type?

var msg pb.MyMessage
if err := proto.Unmarshal(bytes, &msg) {
    return nil, err
}
goTime := msg.Tstamp.AsTime() // get the Go time.Time object
topofstack commented 1 year ago

Thanks for response!

I'm trying to write some kind of an abstract proto parser which should extract fields from different protos/messages

It should work like: 1) User passes MessageDescriptor to the Parser. It acts as a template for incoming binary messages. Now I can inspect internal representation of the incoming byte arrays using protoreflect.

2) User calls something like Parser.parse(byte[]) and in this method passed byte array will be dynamically inspected without knowing it's actual underlying message type. All I know is that this byte array corresponds with the MessageDescriptor i got on the step 1. I need to extract all fields and cast them into Go types for later usage.

The conversion from google.protobuf.Timestamp to time.Time from the code above doesn't look good. So, I'm wondering, if there's a more clear way to perform it.

puellanivis commented 1 year ago

I’m unsure, but a type switch might enable you to go from .Message() to the timestamppb and then you should be able to .AsTime() at that point…

topofstack commented 1 year ago

Sadly, all out-of-the-box conversions are defined in https://github.com/protocolbuffers/protobuf-go/blob/v1.30.0/reflect/protoreflect/value_union.go. So, it looks like the only way is to treat timestamp as a message and parse it by hand like I initially did.