search

twitter / summingbird

Streaming MapReduce with Scalding and Storm
https://twitter.com/summingbird
Apache License 2.0
2.14k stars 267 forks source link

unresolved dependency: com.twitter.common.zookeeper #662

Open nku-ting opened 8 years ago

nku-ting commented 8 years ago

When I execute "./sbt "summingbird-example/run --local" ",the following error occured:

[warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: com.twitter.common.zookeeper#server-set;1.0.103: not found [warn] :: com.twitter.common.zookeeper#client;0.0.60: not found [warn] :: com.twitter.common.zookeeper#group;0.0.78: not found [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] [warn] Note: Unresolved dependencies path: [warn] com.twitter.common.zookeeper:server-set:1.0.103 [warn] +- com.twitter:finagle-serversets_2.10:6.27.0 [warn] +- com.twitter:finagle-cacheresolver_2.10:6.27.0 [warn] +- com.twitter:finagle-memcachedx_2.10:6.27.0 [warn] +- com.twitter:storehaus-memcache_2.10:0.13.0 (/home/sum/summingbird/summingbird-develop/build.sbt#L349) [warn] +- com.twitter:summingbird-example_2.10:0.10.0-SNAPSHOT [warn] com.twitter.common.zookeeper:client:0.0.60 [warn] +- com.twitter:util-zk-common_2.10:6.26.0 [warn] +- com.twitter:finagle-serversets_2.10:6.27.0 [warn] +- com.twitter:finagle-cacheresolver_2.10:6.27.0 [warn] +- com.twitter:finagle-memcachedx_2.10:6.27.0 [warn] +- com.twitter:storehaus-memcache_2.10:0.13.0 (/home/sum/summingbird/summingbird-develop/build.sbt#L349) [warn] +- com.twitter:summingbird-example_2.10:0.10.0-SNAPSHOT [warn] com.twitter.common.zookeeper:group:0.0.78 [warn] +- com.twitter:util-zk-common_2.10:6.26.0 [warn] +- com.twitter:finagle-serversets_2.10:6.27.0 [warn] +- com.twitter:finagle-cacheresolver_2.10:6.27.0 [warn] +- com.twitter:finagle-memcachedx_2.10:6.27.0 [warn] +- com.twitter:storehaus-memcache_2.10:0.13.0 (/home/sum/summingbird/summingbird-develop/build.sbt#L349) [warn] +- com.twitter:summingbird-example_2.10:0.10.0-SNAPSHOT sbt.ResolveException: unresolved dependency: com.twitter.common.zookeeper#server-set;1.0.103: not found unresolved dependency: com.twitter.common.zookeeper#client;0.0.60: not found unresolved dependency: com.twitter.common.zookeeper#group;0.0.78: not found at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:291) at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:188) at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:165) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155) at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:132) at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57) at sbt.IvySbt$$anon$4.call(Ivy.scala:65) at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93) at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78) at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97) at xsbt.boot.Using$.withResource(Using.scala:10) at xsbt.boot.Using$.apply(Using.scala:9) at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58) at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48) at xsbt.boot.Locks$.apply0(Locks.scala:31) at xsbt.boot.Locks$.apply(Locks.scala:28) at sbt.IvySbt.withDefaultLogger(Ivy.scala:65) at sbt.IvySbt.withIvy(Ivy.scala:127) at sbt.IvySbt.withIvy(Ivy.scala:124) at sbt.IvySbt$Module.withModule(Ivy.scala:155) at sbt.IvyActions$.updateEither(IvyActions.scala:165) at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1369) at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1365) at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$87.apply(Defaults.scala:1399) at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$87.apply(Defaults.scala:1397) at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:37) at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1402) at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1396) at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:60) at sbt.Classpaths$.cachedUpdate(Defaults.scala:1419) at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1348) at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1310) at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47) at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40) at sbt.std.Transform$$anon$4.work(System.scala:63) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226) at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17) at sbt.Execute.work(Execute.scala:235) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226) at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159) at sbt.CompletionService$$anon$2.call(CompletionService.scala:28) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

error sbt.ResolveException: unresolved dependency: com.twitter.common.zookeeper#server-set;1.0.103: not found [error] unresolved dependency: com.twitter.common.zookeeper#client;0.0.60: not found [error] unresolved dependency: com.twitter.common.zookeeper#group;0.0.78: not found

Could anyone help me,please? Thank you.

jnievelt commented 8 years ago

FWIW, it should be finding these here:

http://maven.twttr.com/com/twitter/common/zookeeper/server-set/1.0.103/ http://maven.twttr.com/com/twitter/common/zookeeper/client/0.0.60/ http://maven.twttr.com/com/twitter/common/zookeeper/group/0.0.78/

maven.twttr.com should be registered as a resolver here:

https://github.com/twitter/summingbird/blob/6445f050a33b1015731389932649792f65f84a17/build.sbt#L69

However, when I try to run this command on develop/master, I'm getting a different error:

[error] (run-main-0) java.lang.ClassNotFoundException: scala.Function0
java.lang.ClassNotFoundException: scala.Function0
nku-ting commented 8 years ago

Thank you,jnievelt I modified the build.sbt and it worked.As follows:

resolvers ++= Seq( Opts.resolver.sonatypeSnapshots, Opts.resolver.sonatypeReleases, "maven Rpository" at "http://maven.apache.org", "Clojars Repository" at "http://clojars.org/repo", "Conjars Repository" at "http://conjars.org/repo", "oss Repository" at "http://maven.apache.org", "Twitter Maven" at "http://maven.twttr.com", "Twitter Maven Repo" at "http://maven.twttr.com" )

But there are some new errors: [warn] /home/sum/summingbird/summingbird-develop/summingbird-core/src/main/scala/com/twitter/summingbird/OptionalPlatform2.scala:27: match may not be exhaustive. [warn] It would fail on the following input: ValueFlatMappedProducer(, ) [warn] root match { [warn] ^ [warn] /home/sum/summingbird/summingbird-develop/summingbird-core/src/main/scala/com/twitter/summingbird/Platform2.scala:34: match may not be exhaustive. [warn] It would fail on the following input: ValueFlatMappedProducer(, ) [warn] root match { [warn] ^ [warn] there were 42 feature warning(s); re-run with -feature for details [warn] three warnings found [info] Compiling 9 Scala sources to /home/sum/summingbird/summingbird-develop/summingbird-batch/target/scala-2.10/classes... [warn] there were 1 feature warning(s); re-run with -feature for details [warn] one warning found [info] Compiling 3 Scala sources to /home/sum/summingbird/summingbird-develop/summingbird-chill/target/scala-2.10/classes... [info] Compiling 5 Scala sources to /home/sum/summingbird/summingbird-develop/summingbird-client/target/scala-2.10/classes... [info] Compiling 18 Scala sources to /home/sum/summingbird/summingbird-develop/summingbird-online/target/scala-2.10/classes... [warn] /home/sum/summingbird/summingbird-develop/summingbird-online/src/main/scala/com/twitter/summingbird/online/FlatMapOperation.scala:129: Adapting argument list by creating a 2-tuple: this may not be what you want. [warn] signature: ArrowAssoc.->[B](y: B): (A, B) [warn] given arguments: v, x$7 [warn] after adaptation: ArrowAssoc.->((v, x$7): (V, Option[JoinedV])) [warn] val resultFutures = resultList.map { case (k, v) => mres(k).map { k -> (v, ) } }.toIndexedSeq [warn] ^ [warn] /home/sum/summingbird/summingbird-develop/summingbird-online/src/main/scala/com/twitter/summingbird/online/executor/FinalFlatMap.scala:116: Adapting argument list by creating a 2-tuple: this may not be what you want. [warn] signature: ArrowAssoc.->[B](y: B): (A, B) [warn] given arguments: List(state), v [warn] after adaptation: ArrowAssoc.->((List(state), v): (List[S], Value)) [warn] k -> (List(state), v) [warn] ^ [warn] /home/sum/summingbird/summingbird-develop/summingbird-online/src/main/scala/com/twitter/summingbird/online/executor/Summer.scala:106: Adapting argument list by inserting (): this is unlikely to be what you want. [warn] signature: Function1.apply(v1: T1): R [warn] given arguments: [warn] after adaptation: Function1((): Unit) [warn] }.onSuccess { => successHandlerOpt.get.handlerFn.apply() }) [warn] ^ [warn] /home/sum/summingbird/summingbird-develop/summingbird-online/src/main/scala/com/twitter/summingbird/online/executor/FinalFlatMap.scala:133: side-effecting nullary methods are discouraged: suggest defining as def cleanup() instead [warn] override def cleanup { [warn] ^ [warn] /home/sum/summingbird/summingbird-develop/summingbird-online/src/main/scala/com/twitter/summingbird/online/executor/IntermediateFlatMap.scala:49: side-effecting nullary methods are discouraged: suggest defining as def cleanup() instead [warn] override def cleanup { lockedOp.get.close } [warn] ^ [warn] /home/sum/summingbird/summingbird-develop/summingbird-online/src/main/scala/com/twitter/summingbird/online/executor/OperationContainer.scala:29: side-effecting nullary methods are discouraged: suggest defining as def cleanup() instead [warn] def cleanup {} [warn] ^ [warn] /home/sum/summingbird/summingbird-develop/summingbird-online/src/main/scala/com/twitter/summingbird/online/executor/Summer.scala:128: side-effecting nullary methods are discouraged: suggest defining as def cleanup() instead [warn] override def cleanup = Await.result(store.close) [warn] ^ [warn] 7 warnings found [info] Compiling 19 Scala sources and 1 Java source to /home/sum/summingbird/summingbird-develop/summingbird-storm/target/scala-2.10/classes... [warn] /home/sum/summingbird/summingbird-develop/summingbird-storm/src/main/scala/com/twitter/summingbird/storm/StormPlatform.scala:38: imported `StormMetric' is permanently hidden by definition of object StormMetric in package storm [warn] import com.twitter.summingbird.storm.StormMetric [warn] ^ [warn] /home/sum/summingbird/summingbird-develop/summingbird-storm/src/main/scala/com/twitter/summingbird/storm/Producer2FlatMapOperation.scala:37: match may not be exhaustive. [warn] It would fail on the following input: ValueFlatMappedProducer(, ) [warn] p match { [warn] ^ [warn] non-nullary method overrides nullary method [warn] there were 2 feature warning(s); re-run with -feature for details [warn] four warnings found [info] Compiling 5 Scala sources to /home/sum/summingbird/summingbird-develop/summingbird-example/target/scala-2.10/classes... [info] Running com.twitter.summingbird.example.ExeStorm --local SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/root/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/root/.ivy2/cache/org.slf4j/slf4j-jdk14/jars/slf4j-jdk14-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] error java.lang.ClassNotFoundException: scala.Function0 java.lang.ClassNotFoundException: scala.Function0 [trace] Stack trace suppressed: run last summingbird-example/compile:run for the full output. java.lang.RuntimeException: Nonzero exit code: 1 at scala.sys.package$.error(package.scala:27) [trace] Stack trace suppressed: run last summingbird-example/compile:run for the full output. error Nonzero exit code: 1 [error] Total time: 514 s, completed 2016-6-2 10:24:06