search

AbsaOSS / spline-spark-agent

Spline agent for Apache Spark
https://absaoss.github.io/spline/
Apache License 2.0
175 stars 90 forks source link

Getting error while connecting tryng to use Kafka dispatcher #755

Open B09306 opened 8 months ago

B09306 commented 8 months ago

I am trying to use Kafka dispatcher but getting below error in driver logs. I am using version 1.2.0 with spark 3.3.0

23/10/17 08:46:03 INFO SparkLineageInitializer: Initializing Spline Agent... 23/10/17 08:46:03 INFO SparkLineageInitializer: Spline Version: 1.2.0 (rev. f66d36f) 23/10/17 08:46:03 INFO SparkLineageInitializer: Init Type: AUTO (codeless) 23/10/17 08:46:03 INFO SparkLineageInitializer: Init Mode: ENABLED 23/10/17 08:46:03 ERROR DriverDaemon$: XXX Fatal uncaught exception. Terminating driver. java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Producer

Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.producer.Producer

am I missing some configurations?

cerveada commented 8 months ago

add this jar on your classpath:

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>2.4.1</version>
</dependency>