opentraffic / reporter

OTv2: distributed service that matches raw GPS probe data to OSMLR segments and sends anonymized speeds to Datastore
GNU Lesser General Public License v3.0
13 stars 16 forks source link

allow ingest from a Kafka stream #35

Closed drewda closed 7 years ago

drewda commented 7 years ago

Reporter currently can ingest from HTTP requests or from CSV files (which are parsed and send in as HTTP requests).

Many data providers use Kafka to stream location data around their own internal infrastructure. We'll add support for Reporter to act as a consumer to such Kafka streams.

Note that data providers may have different serialization formats for the contents of their Kafka messages. And they'll likely have different key names for lat, lon, accuracy, driver/vehicle ID, vehicle mode, etc. So ideally it's straightforward for each provider to write their own mapping function to parse messages into the appropriate values for use within Reporter.

Requires completion of #36