cosmos / cosmos-proto

Other
37 stars 15 forks source link

Canonical marshalling for imported messages #34

Open fdymylja opened 2 years ago

fdymylja commented 2 years ago

Our deterministic marshalling implementation enforces canonical marshalling for messages. This is achieved because we generated that code with pulsar codegen.

But what happens when we import a message that was not generated with cosmos-proto?

Ex:

message GeneratedWithCosmosProto {
  google.Protobuf.FileDescriptor not_generated_with_cosmos_proto = 1;
  string field = 1;
  map<string, int64> map_field = 2;
  ...
}

How do we detect that the import is not generated with pulsar from marshal codegen, and then how do we marshal it in a canonical way without impacting performance much?

Use cases:

NOTE: We can enforce deterministic marshalling for imported objects, protobuf marshal options supports this, but there is no guarantee that that deterministic marshalling follows https://github.com/cosmos/cosmos-sdk/blob/master/docs/architecture/adr-027-deterministic-protobuf-serialization.md

aaronc commented 2 years ago

So my proposed approach is to create a canonical marshaler in the cosmos-proto runtime that generated code will call on imported messages. Since the official marshaler is reflection based too, there should be a way to keep performance similar.

To simplify this we can probably use the https://pkg.go.dev/google.golang.org/protobuf/reflect/protorange package which also can be used for amino json and maybe unknown field rejection in the future.