Add capability for RethinkDB Connect sink to work with schemaless Kakfa topic

lensesio / stream-reactor

A collection of open source Apache 2.0 Kafka Connector maintained by Lenses.io.

https://lenses.io

Apache License 2.0

1.01k stars 363 forks source link

Add capability for RethinkDB Connect sink to work with schemaless Kakfa topic #62

Closed timchan-lumoslabs closed 8 years ago

timchan-lumoslabs commented 8 years ago

We currently have heterogenous game meta data in JSON appearing on one topic in Kafka. We would like to utilize a RethinkDB Connect sink so that it will just dump this data into a single RethinkDB table. We can not apply a schema on this data at the moment.

dollschasingmen commented 8 years ago

i got u tim

stheppi commented 8 years ago

We have committed the latest changes supporting schemaless records as well as records with schema set to STRING and payload to be a json string. This meets all the functionality of KCQL (picking fields, renaming fields, ignoring fields)