Compilation confused about which Vector to use

[trptcolin]

When building the workshop code in local mode, either via Activator (just downloaded for the first time) or sbt (version 0.13.7, installed via Homebrew), I get the below errors about Vector. If I replace these erroring instances of Vector with scala.collection.immutable.Vector (prompted by the comment next to the first error), then everything seems happy and the tests build just fine.

I don't know the Scala ecosystem well at all, so I suspect I'm missing a build step/configuration. I'm running HotSpot 1.7.0_45-b18 - perhaps my slowness to move to Java 8 could also be an issue?

Happy to submit a patch if this isn't just a PEBKAC.

[error] /Users/colin/Practice/spark-workshop/src/main/scala/sparkworkshop/Matrix4.scala:65: object Vector is not a value
[error]       val outputLines = Vector(          // Scala's Vector, not MLlib's version!
[error]                         ^
[error] /Users/colin/Practice/spark-workshop/src/main/scala/sparkworkshop/NGrams6.scala:87: object Vector is not a value
[error]       val outputLines = Vector(
[error]                         ^
[error] /Users/colin/Practice/spark-workshop/src/main/scala/sparkworkshop/SparkStreaming8Main.scala:76: Vector does not take type parameters
[error]     def mkStreamArgs(argsSeq: Seq[String], newSeq: Vector[String]): Vector[String] =
[error]                                                                     ^
[error] /Users/colin/Practice/spark-workshop/src/main/scala/sparkworkshop/SparkStreaming8Main.scala:76: Vector does not take type parameters
[error]     def mkStreamArgs(argsSeq: Seq[String], newSeq: Vector[String]): Vector[String] =
[error]                                                    ^
[error] /Users/colin/Practice/spark-workshop/src/main/scala/sparkworkshop/SparkStreaming8Main.scala:83: value empty is not a member of object Vector
[error]     val streamArgs = mkStreamArgs(args, Vector.empty[String]).toArray
[error]                                                ^
[error] 6 errors found
[error] (compile:compile) Compilation failed

[deanwampler]

I'm totally mystified by this. I tried to reproduce it with a clean clone of the repo, using both Java 7 and 8, but with no success. Vector is visible automatically in Scala. You could explicitly import it or use the fully qualified name like you did, but that shouldn't be necessary. One possibility is that Spark's own Vector type for machine learning somehow is in scope, but I don't see how that's possible, either. You could try checking that your clone has the latest updates and then running clean and compile within sbt, but I would be surprised if it makes any difference. Perhaps we can review in person this Friday.

[deanwampler]

Doing some housecleaning. I'm going to close this as I suspect it was some sort of environment issue, like the Scala library jar file wasn't on the class path for some reason. I'm not observing this problem currently.

[trptcolin]

I'm revisiting Spark, and when running into this again, observed that:

1. The error goes away if I put the code in question into a package (package trptcolin for example).
2. The error ("object Vector is not a value") comes back, even with the package added, when I add an import java.util.Vector to the package (hinted at by http://stackoverflow.com/questions/9079129/object-is-not-a-value-error-in-scala).

I suspected that maybe some dependency was also doing an import java.util.Vector in the default package, but I wasn't able to reproduce by adding that import into a bunch of other package trptcolin files. This looks to be sort of confirmed by http://www.scala-lang.org/old/node/9497.html - and I don't understand implicits yet, but I'm thinking those could be a next thing to rule out, potentially.

I have a workaround, so not a big deal at all, but thought I'd share in case it helps get closer to an answer or helps anyone else!

[deanwampler]

I just pushed a change that uses the fully-qualified path to Scala's Vector. Hopefully that will fix the issue. Let me know if it appears to work for you.

[trptcolin]

I was able to create a minimal repro case (a 0-dependency Hello World) that proved it's nothing to do with this project, and more related to scala or sbt.

Uninstalling/reinstalling sbt didn't help, nor did deleting ~/.sbt.

Closing since I proved out it's nothing specific about this codebase, sorry for the noise!

[deanwampler]

Weird! I hope you figure out the issue. Thanks for investigating it more.

[trptcolin]

Alright, finally sorted it out. Turned out to be a Java extension that I'd installed in 2012 for a Coursera class. 😿 https://github.com/trptcolin/object-vector-is-not-a-value

[deanwampler]

​Great, I'm glad you isolated it.