search

Workiva / eva

A distributed database-system implementing an entity-attribute-value data-model that is time-aware, accumulative, and atomically consistent
Other
565 stars 25 forks source link

Propagate Metadata from Peer to Transactor #91

Open tylerwilding-wk opened 5 years ago

tylerwilding-wk commented 5 years ago

There's a set of common metadata (correlation-id, in particular) that we would like propagate from the Peer process to the Transactor processes for tracing and logging.

We should standardize our message format to be fully-qualified keyword maps that we can split into metadata and body data to support this.

Ryan Heimbuch:

longer term I think the "solution" would be to just standardize on our message being clojure maps with namespace-qualified-keywords... then at the messaging layer define something like meta-key? and body-key? predicates which are used to split the message-maps... "meta" key-vals go into a headers-field (ie. some specific JMS property), and "body" key-vals are serialized into the body as today

if you want to try moving in this direction, I would suggest that you need to:

  1. define a namespace that contains the proposed message encoding/decoding process (akin to what I did in eva.v2.messaging.jms.alpha.message) where you have: map of ns-qualified-keywords, split the map with explicit meta-keys going into JMS message property and the rest of the kvs going into the body
  2. either copy or extend the MessageProtocol implementation in node.beta... specifically you care about the points that discriminate the message format based on the JMSType message field (which is explicit in node.beta, but implicit in the older releases)
  3. adjust the existing request-response/publish-subscribe call-sites in transactor/peer/indexers to handle both: a. the current message body format: {:throwable? <true|false> :payload <clojure data|serialized-throwable>}\ b. the proposed format: map of ns-qualified-keywords, split the map with explicit meta-keys going into JMS message property and the rest of the kvs going into the body