search

keedio / flume-ng-sql-source

Flume Source to import data from SQL Databases
Apache License 2.0
264 stars 164 forks source link

Kafka topic - Key and partition through Flume #47

Closed josemasuarez closed 6 years ago

josemasuarez commented 6 years ago

Hi,

When I get the data from Oracle in my case, I have to send this data with a specific key and partition to Kafka. Is there any way to specify through flume.conf?

Thanks in advance Regards.

jucaf commented 6 years ago

It's a configuration related to flume kafka sink, not the source. Maybe also you need to set some interceptors.

El 24 nov. 2017 19:56, "José María" notifications@github.com escribió:

Hi,

When I get the data from Oracle in my case, I have to send this data with a specific key and partition to Kafka. Is there any way to specify through flume.conf?

Thanks in advance Regards.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/keedio/flume-ng-sql-source/issues/47, or mute the thread https://github.com/notifications/unsubscribe-auth/AEL271nMmFy3I8dtFdWaDuZv5vkBHmilks5s5xFSgaJpZM4QqHhi .

lazaromedina commented 6 years ago

As jucaf said, you issue is more related to flume-sink than to flume-source component. Check for Apache Flume Kafka sink Best, Luis.