How to convert proto encoded buffer using new protobuf library "google.golang.org/protobuf"

[Tej81r]

Hi team,

We use proto from the protobuf lib in our Golang chain codes to unmarshal the "stub.GetCreator()" result. First, we create an instance of the Serailizedidentity from "github.com/hyperledger/fabric-protos-go/msp" to hold the Unmarshall result from "stub.GetCreator()" and then unmarshall it using proto.

creator, err := stub.GetCreator()
if err != nil {
    return nil, err
}
    //Create a SerializedIdentity to hold Unmarshal GetCreator() result
sId := &msp.SerializedIdentity{}
//Unmarshal the creator from []byte to structure
err := proto.Unmarshal(creator, sId)
if err != nil {
    return nil, err
}

Our product is enterprise edition, and we are not permitted by our company to use deprecated packages. So I tried to use the new protobuf, which is not deprecated and is recommended by the protobuf team, i.e. "google.golang.org/protobuf." I am getting the following error:

"cannot use sId (variable of type msp.SerializedIdentity) as protoreflect.ProtoMessage value in argument to proto.Unmarshal:msp.SerializedIdentity does not implement protoreflect.ProtoMessage (missing method ProtoReflect)compilerInvalidIfaceAssign var sId *msp.SerializedIdentity"

I believe the old "protoc" written by "github.com/golang/protobuf" is used to generate the go code for proto files i.e "identities.pb.go" file. We are unable to unmarshall the serailize data from "stub.Creator()" to msp.SerailizeIdentity using the most recent protobuf "google.golang.org/protobuf/proto" due to "identities.pb.go" file generated using protoc of a deprecated version. Is there any way to use the new protobuf "google.golang.org/protobuf" lib to unmarshal the serialised data from "stub.Creator()" using fabric-protos-go/msp, or is there any plan to update fabric-protos-go in the future so that identities.pb.go generated by "google.golang.org/protobuf"?

[bestbeforetoday]

Go bindings for the Fabric protobufs generated using the newer Go protobuf API are published as hyperledger/fabric-photos-apiv2. See docs for details:

https://hyperledger.github.io/fabric-protos/

It is possible that referencing both versions of the Fabric Go protobuf bindings in the same project might cause a protobuf namespace conflict. You might have to try it and see.

[Tej81r]

Hi,

Yes, I tried that using fabric-protos-go-apiv2. I received the namespace-conflict error saying

panic: proto: file "msp/identities.proto" is already registered See https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict

I cannot remove the fabric-protos-go package as fabric-chaincode-go internally uses it so that I have to use fabric-protos-go in my chain code.

I tried to use fabric-protos-go-apiv2 only for MSP then this namespace conflict occurred.

Now I manually created a package in my chain code and generated the pb.go file using identies.proto with the latest protoc created by google.golang.org/protobuf file by changing the name of the message "serializeIdentity", file name "identities.pb.go" and using it in place of msp.

I believe this is not the correct way. I see solving namespace-conflict has to done from the fabric-protos-go-apiv2 side.

Is there any other way to solve this problem? To un marshall the serialized data into message type serializeIdentity from fabric-protos-go using new protobuf "google.golang.org/protobuf"

Thanks, Raviteja

On Thu, Apr 13, 2023, 5:06 PM Mark S. Lewis @.***> wrote:

Go bindings for the Fabric protobufs generated using the newer Go protobuf API are published as hyperledger/fabric-photos-apiv2. See docs for details:

https://hyperledger.github.io/fabric-protos/

It is possible that referencing both versions of the Fabric Go protobuf bindings in the same project might cause a protobuf namespace conflict https://protobuf.dev/reference/go/faq/#namespace-conflict. You might have to try it and see.

— Reply to this email directly, view it on GitHub https://github.com/hyperledger/fabric-protos/issues/181#issuecomment-1506809911, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVEG7IVBHJYYWTZKWMD2YE3XA7QK3ANCNFSM6AAAAAAWZZNY5A . You are receiving this because you authored the thread.Message ID: @.***>

[bestbeforetoday]

fabric-photos-go-apiv2 is just generated using the protoc compiler from the same protobuf definitions (in this repository) as fabric-protos-go. To generate different output would require different input.

I think modifying the protobuf message definition - either the message or package name - and generating new bindings, as you have done, is probably the most pragmatic approach to get you running. No breaking changes should be introduced into the protobufs so I think you should be pretty safe doing this.

[Tej81r]

Thanks, @bestbeforetoday. Is it planned to update the "pb.go" files in the "fabric-protos-go" package in the future with the new protoc compiler developed by "google.golang.org/protobuf"? Is there another way to overcome the namespace-conflict problem in my chain code without introducing the "pb.go" file by modifying the message and package names?

[bestbeforetoday]

The fabric-protos-go package continues to use the deprecated protoc compiler and Go protocol buffer API precisely because of the incompatibilities you've hit. fabric-protos-go-apiv2 is the equivalent using the new protoc compiler and v2 Go protocol buffer API.

My understanding is that the namespace conflict is a limitation of the global registry used by the Go protobuf implementation. Unless the Go protobuf implementation can fix it, I'm not sure there is other workaround. At least not that I know of, but somebody else might have better information.

[jt-nti]

I think the best long term solution would be to update fabric-chaincode-go to use fabric-protos-go-apiv2. (Would that count as a breaking change?)

Until that happens it probably makes sense to just carry on using fabric-protos-go since the deprecated protobuf API is presumably getting pulled in by fabric-chaincode-go anyway?

[bestbeforetoday]

I agree that updating fabric-chaincode-go (and also fabric-contract-api-go) to use fabric-protos-go-apiv2 would be a good approach. With the potential for mixing different protobuf bindings in the same project to cause failures, perhaps releasing those packages (using the v2 protobuf API) at v2.x would be sensible, with the current (v1.x) releases sticking with fabric-protos-go.

[davidkhala]

@bestbeforetoday I would like to nominate myself to help on the apiv2 creation, while I am also suffering from this issue.

[jt-nti]

@davidkhala it looks like there's already a draft PR for fabric-chaincode-go: hyperledger/fabric-chaincode-go#84

[denyeart]

I commented in the main fabric APIv2 issue.

I would be in favor of updating all fabric repositories to APIv2 before Fabric v3 gets released (including Go chaincode repositories), if somebody tests and successfully proves that having a mixed set of Fabric v2.x orderers and peers (old protobuf) and new v3.x orderers and peers (new APIv2) works seamlessly. Any volunteers?

[bestbeforetoday]

There is now a github.com/hyperledger/fabric-chaincode-go/v2 and an accompanying github.com/hyperledger/fabric-contract-api-go/v2 that both use fabric-protos-go-apiv2, which in turn uses google.golang.org/protobuf. Smart contract implementors can choose whether they want to work with the older or newer protobuf APIs by selecting the appropriate chaincode API version.