search

PacktPublishing / Mastering-Apache-Kafka-2.0

Mastering Apache Kafka 2.0 published by Packt
MIT License
2 stars 1 forks source link

Not compiling because missing dependencies #1

Open alonsoir opened 5 years ago

alonsoir commented 5 years ago

Hi, I just tried to compile with the command: sbt clean package

and i got this output

I have to update spark version.

alonsoir commented 5 years ago

I have to update build.sbt file too.

`name := "SparkJobs"

version := "1.0"

scalaVersion := "2.11.6"

val sparkVersion = "2.4.0"

val flinkVersion = "1.7.2"

val sparkStreamingKafkaVersion = "1.6.3"

resolvers ++= Seq( "Typesafe Releases" at "http://repo.typesafe.com/typesafe/releases/", "apache snapshots" at "http://repository.apache.org/snapshots/", "confluent.io" at "http://packages.confluent.io/maven/" )

libraryDependencies += "javax.ws.rs" % "javax.ws.rs-api" % "2.1" artifacts( Artifact("javax.ws.rs-api", "jar", "jar")) // this is a workaround for https://github.com/jax-rs/api/issues/571

libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % sparkVersion, "org.apache.spark" %% "spark-sql" % sparkVersion,
"org.apache.spark" %% "spark-streaming" % sparkVersion, "org.apache.spark" %% "spark-streaming-kafka" % sparkStreamingKafkaVersion ,
"org.apache.spark" %% "spark-hive" % sparkVersion, "org.apache.flink" %% "flink-connector-kafka-0.10" % flinkVersion, "org.apache.kafka" %% "kafka-streams-scala" % "2.0.0", "io.confluent" % "kafka-streams-avro-serde" % "5.1.0"
)`

But i got this output

I am going to create a project from scratch with this build.sbt without source code folder. I suspect that sbt is going crazy with that path.

alonsoir commented 5 years ago

Still working around. It compiles but it is not working.