Pubsub message signature specification not deterministic

[arnetheduck]

The message signature specification says that signature happens over protobuf encoded message - unfortunately, protobuf encoding is not deterministic, thus it cannot be used for generating a data stream to sign the message: https://developers.google.com/protocol-buffers/docs/encoding#implications

https://github.com/libp2p/specs/tree/b5135fd89e9b62ab94471de89e50f740a83cb004/pubsub#the-message

A possible solution is to specify the data stream explicitly in key-value form where the key and value is encoded according to the wire specification instead: https://developers.google.com/protocol-buffers/docs/encoding#structure - this would allow casual compatibility with what's out there.

[Stebalien]

One option:

Sign:

1. Marshal all fields except the signature/key.
2. Sign that.
3. Marshal the signature/key.
4. Concatenate 1 ++ 3.

Verify:

1. Decode the key/signature into a protobuf.
2. Verify that the signature matches the XXX_unrecognized field.
3. Decode the XXX_unrecognized field.

I mean, ideally we would have used envelopes... but this is the least-bad way to deal with this.

---

For now, the "correct answer" is: encode fields in-order.

[arnetheduck]

For now, the "correct answer" is: encode fields in-order.

this makes sense, but it's more than that: there must be no duplicates, overlong varints etc - it also means that a standard protobuf encoder cannot be used, and therefore the spec must not reference protobuf but rather specify the fields in terms of the underlying encoding.