search

twitter / finatra

Fast, testable, Scala services built on TwitterServer and Finagle
https://twitter.github.io/finatra/
Apache License 2.0
2.27k stars 408 forks source link

sbt.ResolveException: unresolved dependency: com.twitter#scrooge-sbt-plugin;4.20.0-SNAPSHOT: not found #436

Closed christhalinger closed 7 years ago

christhalinger commented 7 years ago

sbt results in an unresolved dependency.

Expected behavior

Successful build.

Actual behavior

[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.twitter#scrooge-sbt-plugin;4.20.0-SNAPSHOT: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn] 
[warn]  Note: Some unresolved dependencies have extra attributes.  Check that these dependencies exist with the requested attributes.
[warn]      com.twitter:scrooge-sbt-plugin:4.20.0-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
[warn] 
[warn]  Note: Unresolved dependencies path:
[warn]      com.twitter:scrooge-sbt-plugin:4.20.0-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13) (/Users/cthalinger/finatra/project/plugins.sbt#L12-13)
[warn]        +- default:finatra-build:0.1-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)

Steps to reproduce the behavior

Clone and run sbt.

yufangong commented 7 years ago

Hi @christhalinger, I think you might in develop branch, you could take a look at Dodo for building against develop branch. Or switch to master and give it another try.

christhalinger commented 7 years ago

I am on:

$ git br
* (HEAD detached at finatra-2.13.0)

Let me try master...

christhalinger commented 7 years ago

master doesn't have a dependency issue but I can't build it either. Is a sbt assembly supposed to work?

All I want to do is build the benchmark-server project.

yufangong commented 7 years ago

@chrisbenincasa can you show us the error log?

christhalinger commented 7 years ago

For top-level or benchmark-server?

yufangong commented 7 years ago

@christhalinger both? more information will be definitely helpful. Thank you!

christhalinger commented 7 years ago

Alright. The top-level output it pretty long but the error seems to be:

java.lang.RuntimeException: deduplicate: different file contents found in the following:
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/com/twitter/finagle-netty4_2.12/7.1.0/finagle-netty4_2.12-7.1.0.jar:BUILD
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/com/twitter/finagle-core_2.12/7.1.0/finagle-core_2.12-7.1.0.jar:BUILD
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/com/twitter/finagle-thrift_2.12/7.1.0/finagle-thrift_2.12-7.1.0.jar:BUILD
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/com/twitter/finagle-mux_2.12/7.1.0/finagle-mux_2.12-7.1.0.jar:BUILD
deduplicate: different file contents found in the following:
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-codec-http/4.1.14.Final/netty-codec-http-4.1.14.Final.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-resolver/4.1.14.Final/netty-resolver-4.1.14.Final.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-codec-socks/4.1.14.Final/netty-codec-socks-4.1.14.Final.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-transport/4.1.14.Final/netty-transport-4.1.14.Final.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-codec/4.1.14.Final/netty-codec-4.1.14.Final.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-transport-native-epoll/4.1.14.Final/netty-transport-native-epoll-4.1.14.Final-linux-x86_64.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-handler-proxy/4.1.14.Final/netty-handler-proxy-4.1.14.Final.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-buffer/4.1.14.Final/netty-buffer-4.1.14.Final.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-handler/4.1.14.Final/netty-handler-4.1.14.Final.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-common/4.1.14.Final/netty-common-4.1.14.Final.jar:META-INF/io.netty.versions.properties
/Users/cthalinger/.coursier/cache/v1/https/repo1.maven.org/maven2/io/netty/netty-transport-native-unix-common/4.1.14.Final/netty-transport-native-unix-common-4.1.14.Final.jar:META-INF/io.netty.versions.properties
    at sbtassembly.Assembly$.applyStrategies(Assembly.scala:140)
    at sbtassembly.Assembly$.x$1$lzycompute$1(Assembly.scala:25)
    at sbtassembly.Assembly$.x$1$1(Assembly.scala:23)
    at sbtassembly.Assembly$.stratMapping$lzycompute$1(Assembly.scala:23)
    at sbtassembly.Assembly$.stratMapping$1(Assembly.scala:23)
    at sbtassembly.Assembly$.inputs$lzycompute$1(Assembly.scala:67)
    at sbtassembly.Assembly$.inputs$1(Assembly.scala:57)
    at sbtassembly.Assembly$.apply(Assembly.scala:83)
    at sbtassembly.Assembly$$anonfun$assemblyTask$1.apply(Assembly.scala:245)
    at sbtassembly.Assembly$$anonfun$assemblyTask$1.apply(Assembly.scala:242)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
christhalinger commented 7 years ago

Here is what I get for benchmark-server:

cthalinger@tw-mbp-cthalinger:~/finatra/examples/benchmark-server$ sbt assembly
[warn] Executing in batch mode.
[warn]   For better performance, hit [ENTER] to switch to interactive mode, or
[warn]   consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading project definition from /Users/cthalinger/finatra/examples/benchmark-server/project
[info] Set current project to benchmark-server (in build file:/Users/cthalinger/finatra/examples/benchmark-server/)
[info] Compiling 2 Scala sources to /Users/cthalinger/finatra/examples/benchmark-server/target/scala-2.12/classes...
[warn] there was one deprecation warning (since 2017-08-23); re-run with -deprecation for details
[warn] one warning found
[info] Compiling 1 Scala source to /Users/cthalinger/finatra/examples/benchmark-server/target/scala-2.12/test-classes...
[info] Including: util-lint_2.12-7.1.0.jar
[info] Including: finatra-slf4j_2.12-2.13.0.jar
[info] Including: slf4j-api-1.7.21.jar
[info] Including: finatra-utils_2.12-2.13.0.jar
[info] Including: libthrift-0.5.0-7.jar
[info] Including: commons-lang-2.6.jar
[info] Including: util-security_2.12-7.1.0.jar
[info] Including: jul-to-slf4j-1.7.21.jar
[info] Including: jcl-over-slf4j-1.7.21.jar
[info] Including: twitter-server_2.12-1.32.0.jar
[info] Including: jackson-databind-2.8.4.jar
[info] Including: scala-reflect-2.12.1.jar
[info] Including: guice-4.1.0.jar
[info] Including: netty-codec-http-4.1.14.Final.jar
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:4: object EmbeddedHttpServer is not a member of package com.twitter.finatra.http
[info] Including: netty-codec-4.1.14.Final.jar
[error] import com.twitter.finatra.http.EmbeddedHttpServer
[info] Including: netty-codec-socks-4.1.14.Final.jar
[error]        ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:6: object Test is not a member of package com.twitter.inject
[error] import com.twitter.inject.Test
[error]        ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:8: not found: type Test
[error] class BenchmarkFeatureTest extends Test {
[error]                                    ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:10: not found: type EmbeddedHttpServer
[error]   val finatraServer = new EmbeddedHttpServer(
[error]                           ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:12: not found: value flags
[error]     flags = Map("http.response.charset.enabled" -> "false")
[error]     ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:14: not found: type EmbeddedHttpServer
[error]   val finagleServer = new EmbeddedHttpServer(new FinagleBenchmarkServer)
[error]                           ^
[info] Including: netty-buffer-4.1.14.Final.jar
[info] Including: commons-codec-1.9.jar
[info] Including: j2objc-annotations-1.1.jar
[info] Including: util-slf4j-jul-bridge_2.12-7.1.0.jar
[info] Including: netty-transport-native-epoll-4.1.14.Final-linux-x86_64.jar
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:16: not found: value test
[error]   test("Benchmark#Servers") {
[error]   ^
[info] Including: finagle-core_2.12-7.1.0.jar
[info] Including: netty-resolver-4.1.14.Final.jar
[info] Including: util-stats_2.12-7.1.0.jar
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:60: value should is not a member of Int
[error]     response.headerMap.size should equal(4)
[error]                             ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:60: not found: value equal
[error]     response.headerMap.size should equal(4)
[error]                                    ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:62: value should is not a member of String
[error]     response.headerMap("Server") should equal(server)
[error]                                  ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:62: not found: value equal
[error]     response.headerMap("Server") should equal(server)
[error]                                         ^
[info] Including: util-tunable_2.12-7.1.0.jar
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:63: value should is not a member of String
[error]     response.headerMap("Content-Type") should equal(contentType)
[error]                                        ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:63: not found: value equal
[error]     response.headerMap("Content-Type") should equal(contentType)
[error]                                               ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:64: value should is not a member of String
[info] Including: finagle-base-http_2.12-7.1.0.jar
[error]     response.headerMap("Content-Length") should equal(contentLength.toString)
[error]                                          ^
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:64: not found: value equal
[error]     response.headerMap("Content-Length") should equal(contentLength.toString)
[error]                                                 ^
[info] Including: finatra-jackson_2.12-2.13.0.jar
[error] /Users/cthalinger/finatra/examples/benchmark-server/src/test/scala/com/twitter/finatra/streaming/BenchmarkFeatureTest.scala:68: value afterAll is not a member of AnyRef
[error]     super.afterAll()
[error]           ^
[error] 16 errors found
[info] Including: inject-modules_2.12-2.13.0.jar
[info] Including: util-collection_2.12-7.1.0.jar
[info] Including: finagle-netty3-http_2.12-7.1.0.jar
[info] Including: util-codec_2.12-7.1.0.jar
[info] Including: servlet-api-2.5.jar
[info] Including: joda-convert-1.2.jar
[info] Including: error_prone_annotations-2.0.18.jar
[info] Including: guice-assistedinject-4.0.jar
[info] Including: netty-handler-4.1.14.Final.jar
[info] Including: netty-handler-proxy-4.1.14.Final.jar
[info] Including: scala-parser-combinators_2.12-1.0.4.jar
[info] Including: finagle-mux_2.12-7.1.0.jar
[info] Including: logback-core-1.1.7.jar
[info] Including: netty-3.10.1.Final.jar
[info] Including: finagle-thrift_2.12-7.1.0.jar
[info] Including: animal-sniffer-annotations-1.14.jar
[info] Including: jackson-module-paranamer-2.8.4.jar
[info] Including: finatra-http_2.12-2.13.0.jar
[info] Including: jackson-datatype-joda-2.8.4.jar
[info] Including: util-slf4j-api_2.12-7.1.0.jar
[info] Including: jackson-module-scala_2.12-2.8.4.jar
[info] Including: log4j-over-slf4j-1.7.21.jar
[info] Including: util-hashing_2.12-7.1.0.jar
[info] Including: netty-transport-4.1.14.Final.jar
[info] Including: finatra-httpclient_2.12-2.13.0.jar
[info] Including: caffeine-2.3.4.jar
[info] Including: jackson-annotations-2.8.4.jar
[info] Including: nscala-time_2.12-2.14.0.jar
[info] Including: util-app_2.12-7.1.0.jar
[info] Including: util-cache_2.12-7.1.0.jar
[info] Including: util-function_2.12-7.1.0.jar
[info] Including: finagle-netty4-http_2.12-7.1.0.jar
[info] Including: netty-codec-http2-4.1.14.Final.jar
[info] Including: util-core_2.12-7.1.0.jar
[info] Including: finagle-zipkin-core_2.12-7.1.0.jar
[info] Including: scalap-2.12.1.jar
[info] Including: inject-slf4j_2.12-2.13.0.jar
[info] Including: scala-guice_2.12-4.1.0.jar
[info] Including: logback-classic-1.1.7.jar
[info] Including: guice-multibindings-4.1.0.jar
[info] Including: paranamer-2.8.jar
[info] Including: inject-app_2.12-2.13.0.jar
[info] Including: javax.inject-1.jar
[info] Including: compiler-0.8.18.jar
[info] Including: finagle-http2_2.12-7.1.0.jar
[info] Including: finagle-init_2.12-7.1.0.jar
[info] Including: jsr305-3.0.1.jar
[info] Including: commons-io-2.4.jar
[info] Including: joda-time-2.9.4.jar
[info] Including: jackson-core-2.8.4.jar
[info] Including: finagle-netty3_2.12-7.1.0.jar
[info] Including: finagle-http_2.12-7.1.0.jar
[info] Including: util-jvm_2.12-7.1.0.jar
[info] Including: inject-core_2.12-2.13.0.jar
[info] Including: finagle-exp_2.12-7.1.0.jar
[info] Including: netty-transport-native-unix-common-4.1.14.Final.jar
[info] Including: util-logging_2.12-7.1.0.jar
[info] Including: scrooge-core_2.12-4.20.0.jar
[info] Including: scala-compiler-2.12.1.jar
[info] Including: inject-server_2.12-2.13.0.jar
[info] Including: netty-common-4.1.14.Final.jar
[info] Including: scala-library-2.12.1.jar
[info] Including: scala-xml_2.12-1.0.6.jar
[info] Including: util-registry_2.12-7.1.0.jar
[info] Including: commons-fileupload-1.3.1.jar
[info] Including: finagle-stats_2.12-7.1.0.jar
[info] Including: guava-23.0.jar
[info] Including: aopalliance-1.0.jar
[info] Including: inject-utils_2.12-2.13.0.jar
[info] Including: finagle-tunable_2.12-7.1.0.jar
[info] Including: finagle-toggle_2.12-7.1.0.jar
[info] Including: finagle-netty4_2.12-7.1.0.jar
[error] (test:compileIncremental) Compilation failed
[error] Total time: 14 s, completed Oct 9, 2017 8:24:58 PM
cacoco commented 7 years ago

@christhalinger just FYI, there's information in the project README.md for benchmark-server and in the Contributing.md for how to work from a non-release/non-master branch (which attempts to explain why you can't just run sbt).

In any case, it looks like the build.sbt has unfortunately gotten out of date for the examples. Sorry. We have an internal ticket to audit these as they are only materialized at release.

You should be able to just build the assembly without the local test:

[cacoco@local benchmark-server (master)]$ sbt 'set test in assembly := {}' clean assembly
...
[info] Packaging ~/finatra/examples/benchmark-server/target/scala-2.12/benchmark-server-assembly-2.13.0.jar ...
[info] Done packaging.
[success] Total time: 36 s, completed Oct XX, 2017 XX:XX:XX PM

[cacoco@local benchmark-server (master)]$ jar tvf target/scala-2.12/benchmark-server-assembly-2.13.0.jar | grep FinatraBenchmarkServerMain
   541 Tue Oct XX XX:XX:XX PDT 2017 com/twitter/finatra/http/benchmark/FinatraBenchmarkServerMain$.class
  5527 Tue Oct XX XX:XX:XX PDT 2017 com/twitter/finatra/http/benchmark/FinatraBenchmarkServerMain.class

Or if you are feeling more intrepid: you can always run the examples from the root project in the develop branch (using the sbt script in the base of the project). However, you will need to run a Dodo build first in order to build Finatra's Twitter library dependencies. E.g.,

[cacoco@local finatra (develop)]$ curl -s https://raw.githubusercontent.com/twitter/dodo/develop/bin/build | bash -s -- --no-test finatra
[INFO] Determining projects...
[INFO] ------------------------------------------------------------------------
[INFO] Dodo Build Order:
[INFO]   util
[INFO]   scroogecore
[INFO]   finagle
[INFO]   scrooge
[INFO]   twitter-server
...
<GO GET A COFFEE, COME BACK SOMETIME LATER>
...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1677 s
[INFO] Finished at: 2017TXX:XX:XX-0700
[INFO] Dodo version v0.0.1
[INFO] ------------------------------------------------------------------------

Then run the assembly command on the benchmarkServer project defined in the root build.sbt:

[cacoco@local finatra (develop)]$ ./sbt benchmarkServer/assembly
...
[info] Packaging ~/finatra/examples/benchmark-server/target/scala-2.12/benchmark-server-assembly-2.14.0-SNAPSHOT.jar ...
[info] Done packaging.
[success] Total time: 154 s, completed Oct XX, 2017 XX:XX:XX PM

We'll get the build.sbt files updated and working before the next release.

Thanks!

christhalinger commented 7 years ago

I read the README and tried to follow the steps but didn't work.

Yes, 'set test in assembly := {}' did the trick. Thanks!

cacoco commented 7 years ago

Examples are now always part of the root build.sbt: 0299399095c67b2fa72c3945c27709d915688ac3