search

Esri / kafka-for-geoevent

ArcGIS GeoEvent Server sample Kafka connectors for connecting to Apache Kafka message servers.
Apache License 2.0
14 stars 10 forks source link

Kafka Message Key discarded; Channel ID instead? #12

Open amihalik opened 7 years ago

amihalik commented 7 years ago

A Kafka Message (or Record) is composed of a Key, Value, and Timestamp and there are currently use cases where data in the Message Key is needed to properly process the Message Value. In these cases, the GeoEvent Adapter will require both the Message Key and Message Value to create a GeoEvent.

The current KafkaInboundTransport drops the Message Key on the floor. I propose that the Message Key is used as the Channel ID. If we pass the Message Key as the Channel ID, the GeoEvent Adapter will have both the Message Key and Value when it deserializes the GeoEvent.