RuckusWirelessIL / pentaho-kafka-producer

Apache Kafka producer step plug-in for Pentaho Kettle.
Apache License 2.0
45 stars 33 forks source link

Producer doesnt work when given parameters from a call job #5

Closed pricecarl closed 8 years ago

pricecarl commented 9 years ago

Hello,

I have a job with passes two parameters down to a transform which is pulling data from a database and putting it on to Kafka topic using your producer.

It works great when I run the transform on its own defaulting the parameter values but when I pass them in from the calling job it fails to connect to the topic.

2015/10/08 17:00:40 - Apache Kafka Producer.0 - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Unexpected error
2015/10/08 17:00:40 - Apache Kafka Producer.0 - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
2015/10/08 17:00:40 - Apache Kafka Producer.0 -     at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
2015/10/08 17:00:40 - Apache Kafka Producer.0 -     at kafka.producer.Producer.send(Producer.scala:77)
2015/10/08 17:00:40 - Apache Kafka Producer.0 -     at kafka.javaapi.producer.Producer.send(Producer.scala:33)
2015/10/08 17:00:40 - Apache Kafka Producer.0 -     at com.ruckuswireless.pentaho.kafka.producer.KafkaProducerStep.processRow(KafkaProducerStep.java:116)
2015/10/08 17:00:40 - Apache Kafka Producer.0 -     at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2015/10/08 17:00:40 - Apache Kafka Producer.0 -     at java.lang.Thread.run(Thread.java:745)
2015/10/08 17:00:40 - EPO_DatabaseToKafka - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Errors detected!

The parameters are:

KAFKABROKERLIST which contains 192.168.1.100:9092,192.168.1.101:9092 EPOKAFKATOPIC which contains EPO

Please can you look into this :)

Many thanks

spektom commented 9 years ago

Can you attach transformation sample?

spektom commented 8 years ago

Can't reproduce.