camunda-community-hub / zeebe-exporter-protobuf

Protobuf schema definition for Zeebe exporters to exchange/transmit records
Apache License 2.0
8 stars 6 forks source link

Decoding Support and Use of Any #310

Open jamulv opened 9 months ago

jamulv commented 9 months ago

Normally I would approach the proto modelling of a set of messages by defining a oneOf as opposed to using an Any type.

The benefits in doing this would be:

  1. Any uses more bytes than the actual message content as it encodes a long typeUrl (certainly true for small messages)
  2. Any does not automatically decode - it takes a significant amount of error prone code to decode the current any output

In applying this we would have two options:

  1. add a top level oneOf for all record types (slightly more compatible with existing code; as this would just add an additional encode overload
  2. re-model so that you mirror the Zeebe structure: Record has a RecordValue (oneOf) this would have the benefit that something can generically process record level metadata (this would break existing api compatibility)
saig0 commented 9 months ago

@jamulv thank you for raising this. :+1: This approach sounds interesting.

If we change anything in the schema, we would need to make sure that we don't break the compatibility.

Feel free to contribute a suggestion. :rocket:

My main usage of the exporter is in combination with the Hazelcast exporter. You can see here how it reads the generic records.