search

huawei-noah / streamDM

Stream Data Mining Library for Spark Streaming
http://streamdm.noahlab.com.hk/
Apache License 2.0
492 stars 147 forks source link

Build failure (unresolved dependency) #102

Closed kination closed 6 years ago

kination commented 6 years ago

Bug Report template

Expected behavior

Success build with command sbt package

Observed behavior

<< What results have you obtained? Is there an error message? If so, please paste it in here (not all issues produce error messages) >> While following up the tutorial, build stoped with failure.

Steps to reproduce the issue

Build fail with log with command sbt package

[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.typesafe.sbteclipse#sbteclipse-plugin;4.0.0: not found
[error] unresolved dependency: com.github.mpeltonen#sbt-idea;1.6.0: not found
[error] unresolved dependency: org.netbeans.nbsbt#nbsbt-plugin;1.1.4: not found
[error] unresolved dependency: com.orrsella#sbt-sublime;1.1.1: not found
[error]     at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve(IvyActions.scala:332)
[error]     at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEither$1(IvyActions.scala:208)
[error]     at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:239)
[error]     at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:204)
[error]     at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error]     at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error]     at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
[error]     at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
[error]     at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
[error]     at xsbt.boot.Using$.withResource(Using.scala:10)
[error]     at xsbt.boot.Using$.apply(Using.scala:9)
[error]     at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
[error]     at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
[error]     at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error]     at xsbt.boot.Locks$.apply(Locks.scala:28)
[error]     at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error]     at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:199)
[error]     at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:196)
[error]     at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:238)
[error]     at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyActions.scala:193)
[error]     at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyDependencyResolution.scala:20)
[error]     at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:56)
[error]     at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:45)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:93)
[error]     at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(LibraryManagement.scala:106)
[error]     at scala.util.control.Exception$Catch.apply(Exception.scala:224)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:106)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:89)
[error]     at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
[error]     at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:120)
[error]     at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2519)
[error]     at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error]     at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error]     at sbt.std.Transform$$anon$4.work(System.scala:67)
[error]     at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error]     at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error]     at sbt.Execute.work(Execute.scala:278)
[error]     at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error]     at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error]     at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error]     at java.lang.Thread.run(Thread.java:745)
[error] (update) sbt.librarymanagement.ResolveException: unresolved dependency: com.typesafe.sbteclipse#sbteclipse-plugin;4.0.0: not found
[error] unresolved dependency: com.github.mpeltonen#sbt-idea;1.6.0: not found
[error] unresolved dependency: org.netbeans.nbsbt#nbsbt-plugin;1.1.4: not found
[error] unresolved dependency: com.orrsella#sbt-sublime;1.1.1: not found
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? i

Infrastructure details

hmgomes commented 6 years ago

Dear @djKooks

What is your Spark version ?

Best Regards, Heitor

kination commented 6 years ago

@hmgomes I'm testing on:

MacOS 10.13
Spark 2.3.0

Is it not supporting yet?

hmgomes commented 6 years ago

@djKooks yes, you will need to use Spark 2.2. However, it should be an easy fix to upgrade it to Spark 2.3, I will go ahead and create a pull request for that as soon as I am back to the office

kination commented 6 years ago

@hmgomes okay, thanks. But it seems there are problem loading:

com.typesafe.sbteclipse#sbteclipse-plugin;4.0.0: not found
[error] unresolved dependency: com.github.mpeltonen#sbt-idea;1.6.0: not found
[error] unresolved dependency: org.netbeans.nbsbt#nbsbt-plugin;1.1.4: not found
[error] unresolved dependency: com.orrsella#sbt-sublime;1.1.1: not found

Could you also look on this?

hmgomes commented 6 years ago

@djKooks I've created a pull request related to this issue.

Regarding the other issue you've just mentioned, it looks like an issue with sbt configuration. I believe the folk in https://github.com/sbt/sbteclipse/issues/315 had the same issue and there are some workarounds (see latest replies). Please see if that works for you, if not I will try to replicate the error later and see what we can do about it.

@hmgomes okay, thanks. But it seems there are problem loading:

com.typesafe.sbteclipse#sbteclipse-plugin;4.0.0: not found
[error] unresolved dependency: com.github.mpeltonen#sbt-idea;1.6.0: not found
[error] unresolved dependency: org.netbeans.nbsbt#nbsbt-plugin;1.1.4: not found
[error] unresolved dependency: com.orrsella#sbt-sublime;1.1.1: not found

Could you also look on this?

kination commented 6 years ago

@hmgomes thanks for help. Unfortunately, it seems not working still. I updated sbteclipse-plugin to 5.2.4 as in link, but other 3 still not works.

If I ignore unresolved dependency:

[error]     at java.lang.Thread.run(Thread.java:745)
[error] (update) sbt.librarymanagement.ResolveException: unresolved dependency: com.github.mpeltonen#sbt-idea;1.6.0: not found
[error] unresolved dependency: org.netbeans.nbsbt#nbsbt-plugin;1.1.4: not found
[error] unresolved dependency: com.orrsella#sbt-sublime;1.1.1: not found
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? i
[warn] Ignoring load failure: no project loaded.
[error] java.lang.RuntimeException: Session not initialized.
[error]     at scala.sys.package$.error(package.scala:27)
[error]     at sbt.Project$.$anonfun$getOrError$1(Project.scala:442)
[error]     at scala.Option.getOrElse(Option.scala:121)
[error]     at sbt.Project$.getOrError(Project.scala:442)
[error]     at sbt.Project$.session(Project.scala:448)
[error]     at sbt.Project$.extract(Project.scala:453)
[error]     at sbt.BuiltinCommands$.notifyUsersAboutShell(Main.scala:928)
[error]     at sbt.BuiltinCommands$.$anonfun$notifyUsersAboutShell$3(Main.scala:937)
[error]     at sbt.Command$.$anonfun$command$2(Command.scala:91)
[error]     at sbt.Command$.process(Command.scala:181)
[error]     at sbt.MainLoop$.processCommand(MainLoop.scala:151)
[error]     at sbt.MainLoop$.$anonfun$next$2(MainLoop.scala:139)
[error]     at sbt.State$$anon$1.runCmd$1(State.scala:246)
[error]     at sbt.State$$anon$1.process(State.scala:250)
[error]     at sbt.MainLoop$.$anonfun$next$1(MainLoop.scala:139)
[error]     at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error]     at sbt.MainLoop$.next(MainLoop.scala:139)
[error]     at sbt.MainLoop$.run(MainLoop.scala:132)
[error]     at sbt.MainLoop$.$anonfun$runWithNewLog$1(MainLoop.scala:110)
[error]     at sbt.io.Using.apply(Using.scala:22)
[error]     at sbt.MainLoop$.runWithNewLog(MainLoop.scala:104)
[error]     at sbt.MainLoop$.runAndClearLast(MainLoop.scala:59)
[error]     at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:44)
[error]     at sbt.MainLoop$.runLogged(MainLoop.scala:35)
[error]     at sbt.StandardMain$.runManaged(Main.scala:138)
[error]     at sbt.xMain.run(Main.scala:89)
[error]     at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
[error]     at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
[error]     at xsbt.boot.Launch$.run(Launch.scala:109)
[error]     at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
[error]     at xsbt.boot.Launch$.launch(Launch.scala:117)
[error]     at xsbt.boot.Launch$.apply(Launch.scala:18)
[error]     at xsbt.boot.Boot$.runImpl(Boot.scala:56)
[error]     at xsbt.boot.Boot$.main(Boot.scala:18)
[error]     at xsbt.boot.Boot.main(Boot.scala)
[error] Session not initialized.

it forced to close. Could you find something more?

hmgomes commented 6 years ago

@hmgomes thanks for help. Unfortunately, it seems not working still. I updated sbteclipse-plugin to 5.2.4 as in link, but other 3 still not works.

If I ignore unresolved dependency:

[error]   at java.lang.Thread.run(Thread.java:745)
[error] (update) sbt.librarymanagement.ResolveException: unresolved dependency: com.github.mpeltonen#sbt-idea;1.6.0: not found
[error] unresolved dependency: org.netbeans.nbsbt#nbsbt-plugin;1.1.4: not found
[error] unresolved dependency: com.orrsella#sbt-sublime;1.1.1: not found
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? i
[warn] Ignoring load failure: no project loaded.
[error] java.lang.RuntimeException: Session not initialized.
[error]   at scala.sys.package$.error(package.scala:27)
[error]   at sbt.Project$.$anonfun$getOrError$1(Project.scala:442)
[error]   at scala.Option.getOrElse(Option.scala:121)
[error]   at sbt.Project$.getOrError(Project.scala:442)
[error]   at sbt.Project$.session(Project.scala:448)
[error]   at sbt.Project$.extract(Project.scala:453)
[error]   at sbt.BuiltinCommands$.notifyUsersAboutShell(Main.scala:928)
[error]   at sbt.BuiltinCommands$.$anonfun$notifyUsersAboutShell$3(Main.scala:937)
[error]   at sbt.Command$.$anonfun$command$2(Command.scala:91)
[error]   at sbt.Command$.process(Command.scala:181)
[error]   at sbt.MainLoop$.processCommand(MainLoop.scala:151)
[error]   at sbt.MainLoop$.$anonfun$next$2(MainLoop.scala:139)
[error]   at sbt.State$$anon$1.runCmd$1(State.scala:246)
[error]   at sbt.State$$anon$1.process(State.scala:250)
[error]   at sbt.MainLoop$.$anonfun$next$1(MainLoop.scala:139)
[error]   at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error]   at sbt.MainLoop$.next(MainLoop.scala:139)
[error]   at sbt.MainLoop$.run(MainLoop.scala:132)
[error]   at sbt.MainLoop$.$anonfun$runWithNewLog$1(MainLoop.scala:110)
[error]   at sbt.io.Using.apply(Using.scala:22)
[error]   at sbt.MainLoop$.runWithNewLog(MainLoop.scala:104)
[error]   at sbt.MainLoop$.runAndClearLast(MainLoop.scala:59)
[error]   at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:44)
[error]   at sbt.MainLoop$.runLogged(MainLoop.scala:35)
[error]   at sbt.StandardMain$.runManaged(Main.scala:138)
[error]   at sbt.xMain.run(Main.scala:89)
[error]   at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
[error]   at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
[error]   at xsbt.boot.Launch$.run(Launch.scala:109)
[error]   at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
[error]   at xsbt.boot.Launch$.launch(Launch.scala:117)
[error]   at xsbt.boot.Launch$.apply(Launch.scala:18)
[error]   at xsbt.boot.Boot$.runImpl(Boot.scala:56)
[error]   at xsbt.boot.Boot$.main(Boot.scala:18)
[error]   at xsbt.boot.Boot.main(Boot.scala)
[error] Session not initialized.

it forced to close. Could you find something more?

Well, we are converging to a solution, it is now a runtime error :) Let's see: 1) were you able to sbt package and generate the jar file for the project? 2) which command line triggered this error?

Cheers, Heitor

hmgomes commented 6 years ago

Another thing, can you try with Scala 2.11.8?

kination commented 6 years ago

@hmgomes thanks for help! 1, 2. No, this issue happens in sbt package command.

This has been also tested in Scala 2.11.10. Is it different with 2.11.8?

hmgomes commented 6 years ago

@djKooks Either of the Scala versions should work fine, but I was thinking of something different. What is your sbt version?

kination commented 6 years ago

@hmgomes mine is sbt 1.2.0 .

hmgomes commented 6 years ago

@djKooks Can you try sbt 0.13?

kination commented 6 years ago

@hmgomes oh, you're right! It works well in sbt 0.13.16. I'm not sure of the reason, but it will be good to add comment about this.

Thanks for help~

hmgomes commented 6 years ago

Thanks @djKooks for testing it. I will add a comment about it in the readme.

Best wishes,

Heitor