RedHatInsights / insights-host-inventory

GNU General Public License v2.0
30 stars 79 forks source link

Double encoding of egress events #406

Open Glutexo opened 5 years ago

Glutexo commented 5 years ago

The events produced on the platform.inventory.host-egress topic are currently double-encoded to JSON.

  1. First encoding happens in the _app.queue.egress.build_hostevent function, where Marshmallow Schema is used to validate and encode the event dictionary. This produces a JSON string.
  2. Second encoding happens in the _app.queue.egress.KafkaEventProducer.writeevent method by plain json.dumps.

The result is a JSON string (not a dictionary) containing properly-escaped JSON dictionary literal. "{\"type\":\"created\"}" instead of {"type":"created"}. That means that if a client decodes the message, it doesn’t get the dictionary, but a string, which must be decoded again.

Possible solutions:

Passing down the Marshmallow object is an easy and quite ok solution.

I am disappoint that there are no tests for the message format. A bug like this would be probably caught by a test.

Glutexo commented 5 years ago

This is probably fixed by https://github.com/RedHatInsights/insights-host-inventory/commit/ff912a5d5ca3780c400a2838715103653435f72f. As far as I know, there are no tests that would confirm this though.