search

scala / bug

Scala 2 bug reports only. Please, no questions — proper bug reports only.
https://scala-lang.org
232 stars 21 forks source link

for-yield comprehension or flatMap gives an IOException: error: Operation not permitted #9132

Closed scabug closed 6 years ago

scabug commented 9 years ago

When I try to compile following code, with a for-comprehension of mostly Try (tries). UploadResult is a case class

for {
     _ <- markerUtil.uploadingStarted(source, destPath, files.map(_.path)) // Try[Unit]
     inputStream <- createInputStream(files) // Try[InputStream]
     _ <- store.upload(destPath, inputStream) // Try[Unit]
     _ <- markerUtil.uploadingDone(source) // Try[Unit]
     _ <- deleteFiles(files) // Try[Unit]
     _ <- markerUtil.uploadedDone(source) // Try[Unit]
   } yield ()

I get following output. Strangely the error happens in a completely different part of the code (in another package).

[INFO] /vagrant/core-compute-jobs-factstore/src/main/scala:-1: info: compiling
[INFO] Compiling 14 source files to /vagrant/core-compute-jobs-factstore/target/classes at 1422892024304
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[ERROR] error:
[INFO]      while compiling: /vagrant/core-compute-jobs-factstore/src/main/scala/com/example/core/compute/jobs/factstore/util/FileSystemFactory.scala
[INFO]         during phase: jvm
[INFO]      library version: version 2.10.4
[INFO]     compiler version: version 2.10.4
[INFO]   reconstructed args: -classpath /vagrant/core-compute-jobs-factstore/target/classes:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-client/2.0.0-mr1-cdh4.4.0/hadoop-client-2.0.0-mr1-cdh4.4.0.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-common/2.0.0-cdh4.4.0/hadoop-common-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-annotations/2.0.0-cdh4.4.0/hadoop-annotations-2.0.0-cdh4.4.0.jar:/usr/lib/jvm/java-7-openjdk-amd64/jre/../lib/tools.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/home/vagrant/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/home/vagrant/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:/home/vagrant/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/home/vagrant/.m2/repository/commons-lang/commons-lang/2.5/commons-lang-2.5.jar:/home/vagrant/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/home/vagrant/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/home/vagrant/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/home/vagrant/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/home/vagrant/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/home/vagrant/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:/home/vagrant/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:/home/vagrant/.m2/repository/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5.jar:/home/vagrant/.m2/repository/org/apache/avro/avro/1.7.4/avro-1.7.4.jar:/home/vagrant/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/home/vagrant/.m2/repository/org/tukaani/xz/1.0/xz-1.0.jar:/home/vagrant/.m2/repository/com/google/protobuf/protobuf-java/2.4.0a/protobuf-java-2.4.0a.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-auth/2.0.0-cdh4.4.0/hadoop-auth-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/com/jcraft/jsch/0.1.42/jsch-0.1.42.jar:/home/vagrant/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5-cdh4.4.0/zookeeper-3.4.5-cdh4.4.0.jar:/home/vagrant/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.0.0-cdh4.4.0/hadoop-hdfs-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar:/home/vagrant/.m2/repository/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar:/home/vagrant/.m2/repository/asm/asm/3.1/asm-3.1.jar:/home/vagrant/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-core/2.0.0-mr1-cdh4.4.0/hadoop-core-2.0.0-mr1-cdh4.4.0.jar:/home/vagrant/.m2/repository/hsqldb/hsqldb/1.8.0.10/hsqldb-1.8.0.10.jar:/home/vagrant/.m2/repository/com/amazonaws/aws-java-sdk/1.6.4/aws-java-sdk-1.6.4.jar:/home/vagrant/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/home/vagrant/.m2/repository/commons-codec/commons-codec/1.3/commons-codec-1.3.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.1.1/jackson-core-2.1.1.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.1.1/jackson-databind-2.1.1.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.1.1/jackson-annotations-2.1.1.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar:/home/vagrant/.m2/repository/org/apache/spark/spark-core_2.10/1.1.1/spark-core_2.10-1.1.1.jar:/home/vagrant/.m2/repository/net/java/dev/jets3t/jets3t/0.7.1/jets3t-0.7.1.jar:/home/vagrant/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-recipes/2.4.0/curator-recipes-2.4.0.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-framework/2.4.0/curator-framework-2.4.0.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-client/2.4.0/curator-client-2.4.0.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-plus/8.1.14.v20131031/jetty-plus-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.transaction/1.1.1.v201105210645/javax.transaction-1.1.1.v201105210645.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-webapp/8.1.14.v20131031/jetty-webapp-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-xml/8.1.14.v20131031/jetty-xml-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-servlet/8.1.14.v20131031/jetty-servlet-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-jndi/8.1.14.v20131031/jetty-jndi-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.mail.glassfish/1.4.1.v201005082020/javax.mail.glassfish-1.4.1.v201005082020.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.activation/1.1.0.v201105071233/javax.activation-1.1.0.v201105071233.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-security/8.1.14.v20131031/jetty-security-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-util/8.1.14.v20131031/jetty-util-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-server/8.1.14.v20131031/jetty-server-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-continuation/8.1.14.v20131031/jetty-continuation-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-http/8.1.14.v20131031/jetty-http-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-io/8.1.14.v20131031/jetty-io-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/com/google/guava/guava/14.0.1/guava-14.0.1.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/home/vagrant/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/home/vagrant/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar:/home/vagrant/.m2/repository/org/slf4j/jul-to-slf4j/1.7.5/jul-to-slf4j-1.7.5.jar:/home/vagrant/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.5/jcl-over-slf4j-1.7.5.jar:/home/vagrant/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/home/vagrant/.m2/repository/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar:/home/vagrant/.m2/repository/com/ning/compress-lzf/1.0.0/compress-lzf-1.0.0.jar:/home/vagrant/.m2/repository/org/xerial/snappy/snappy-java/1.0.5.3/snappy-java-1.0.5.3.jar:/home/vagrant/.m2/repository/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar:/home/vagrant/.m2/repository/com/twitter/chill_2.10/0.3.6/chill_2.10-0.3.6.jar:/home/vagrant/.m2/repository/com/twitter/chill-java/0.3.6/chill-java-0.3.6.jar:/home/vagrant/.m2/repository/commons-net/commons-net/2.2/commons-net-2.2.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-remote_2.10/2.2.3-shaded-protobuf/akka-remote_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-actor_2.10/2.2.3-shaded-protobuf/akka-actor_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/io/netty/netty/3.6.6.Final/netty-3.6.6.Final.jar:/home/vagrant/.m2/repository/org/spark-project/protobuf/protobuf-java/2.4.1-shaded/protobuf-java-2.4.1-shaded.jar:/home/vagrant/.m2/repository/org/uncommons/maths/uncommons-maths/1.2.2a/uncommons-maths-1.2.2a.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-slf4j_2.10/2.2.3-shaded-protobuf/akka-slf4j_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/org/json4s/json4s-jackson_2.10/3.2.10/json4s-jackson_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/json4s/json4s-core_2.10/3.2.10/json4s-core_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/json4s/json4s-ast_2.10/3.2.10/json4s-ast_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/scala-lang/scalap/2.10.0/scalap-2.10.0.jar:/home/vagrant/.m2/repository/colt/colt/1.2.0/colt-1.2.0.jar:/home/vagrant/.m2/repository/concurrent/concurrent/1.3.4/concurrent-1.3.4.jar:/home/vagrant/.m2/repository/org/apache/mesos/mesos/0.18.1/mesos-0.18.1-shaded-protobuf.jar:/home/vagrant/.m2/repository/io/netty/netty-all/4.0.23.Final/netty-all-4.0.23.Final.jar:/home/vagrant/.m2/repository/com/clearspring/analytics/stream/2.7.0/stream-2.7.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-core/3.0.0/metrics-core-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-jvm/3.0.0/metrics-jvm-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-json/3.0.0/metrics-json-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-graphite/3.0.0/metrics-graphite-3.0.0.jar:/home/vagrant/.m2/repository/org/tachyonproject/tachyon-client/0.5.0/tachyon-client-0.5.0.jar:/home/vagrant/.m2/repository/org/tachyonproject/tachyon/0.5.0/tachyon-0.5.0.jar:/home/vagrant/.m2/repository/org/spark-project/pyrolite/2.0.1/pyrolite-2.0.1.jar:/home/vagrant/.m2/repository/net/sf/py4j/py4j/0.8.2.1/py4j-0.8.2.1.jar:/home/vagrant/.m2/repository/joda-time/joda-time/2.2/joda-time-2.2.jar:/home/vagrant/.m2/repository/org/apache/httpcomponents/httpclient/4.2.6/httpclient-4.2.6.jar:/home/vagrant/.m2/repository/org/apache/httpcomponents/httpcore/4.2.5/httpcore-4.2.5.jar:/home/vagrant/.m2/repository/com/esotericsoftware/kryo/kryo/2.22/kryo-2.22.jar:/home/vagrant/.m2/repository/com/typesafe/config/1.2.1/config-1.2.1.jar:/home/vagrant/.m2/repository/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/core-common-datatypes-domain-1.6.0-SNAPSHOT.jar:/home/vagrant/.m2/repository/org/apache/thrift/libthrift/0.9.1/libthrift-0.9.1.jar:/home/vagrant/.m2/repository/com/twitter/scrooge-core_2.10/3.12.3/scrooge-core_2.10-3.12.3.jar:/home/vagrant/.m2/repository/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/core-compute-util-spark-metrics-1.0.0-SNAPSHOT.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-ganglia/3.0.1/metrics-ganglia-3.0.1.jar:/home/vagrant/.m2/repository/info/ganglia/gmetric4j/gmetric4j/1.0.3/gmetric4j-1.0.3.jar:/home/vagrant/.m2/repository/org/acplt/oncrpc/1.0.7/oncrpc-1.0.7.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-reflect/2.10.3/scala-reflect-2.10.3.jar:/home/vagrant/.m2/repository/junit/junit/4.11/junit-4.11.jar:/home/vagrant/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-compiler/2.10.3/scala-compiler-2.10.3.jar -d /vagrant/core-compute-jobs-factstore/target/classes
[INFO]
[INFO]   last tree to typer: Literal(Constant(com.example.core.compute.jobs.factstore.source.SourceDirectory))
[INFO]               symbol: null
[INFO]    symbol definition: null
[INFO]                  tpe: Class(classOf[com.example.core.compute.jobs.factstore.source.SourceDirectory])
[INFO]        symbol owners:
[INFO]       context owners: class FileSystemFactory -> package util
[INFO]
[INFO] == Enclosing template or block ==
[INFO]
[INFO] Template( // val <local FileSystemFactory>: <notype> in class FileSystemFactory, tree.tpe=com.example.core.compute.jobs.factstore.util.FileSystemFactory
[INFO]   "java.lang.Object", "scala.Serializable" // parents
[INFO]   ValDef(
[INFO]     private
[INFO]     "_"
[INFO]     <tpt>
[INFO]     <empty>
[INFO]   )
[INFO]   // 3 statements
[INFO]   ValDef( // private[this] val uri: String in class FileSystemFactory
[INFO]     private <local> <paramaccessor> <triedcooking>
[INFO]     "uri"
[INFO]     <tpt> // tree.tpe=String
[INFO]     <empty>
[INFO]   )
[INFO]   DefDef( // def get(): org.apache.hadoop.fs.FileSystem in class FileSystemFactory
[INFO]     <method> <triedcooking>
[INFO]     "get"
[INFO]     []
[INFO]     List(Nil)
[INFO]     <tpt> // tree.tpe=org.apache.hadoop.fs.FileSystem
[INFO]     Apply( // def get(x$1: java.net.URI,x$2: org.apache.hadoop.conf.Configuration): org.apache.hadoop.fs.FileSystem in object FileSystem, tree.tpe=org.apache.hadoop.fs.FileSystem
[INFO]       "org"."apache"."hadoop"."fs"."FileSystem"."get" // def get(x$1: java.net.URI,x$2: org.apache.hadoop.conf.Configuration): org.apache.hadoop.fs.FileSystem in object FileSystem, tree.tpe=(x$1: java.net.URI, x$2: org.apache.hadoop.conf.Configuration)org.apache.hadoop.fs.FileSystem
[INFO]       // 2 arguments
[INFO]       Apply( // def <init>(x$1: String): java.net.URI in class URI, tree.tpe=java.net.URI
[INFO]         new java.net.URI."<init>" // def <init>(x$1: String): java.net.URI in class URI, tree.tpe=(x$1: String)java.net.URI
[INFO]         FileSystemFactory.this."uri" // private[this] val uri: String in class FileSystemFactory, tree.tpe=String
[INFO]       )
[INFO]       Apply( // def <init>(): org.apache.hadoop.conf.Configuration in class Configuration, tree.tpe=org.apache.hadoop.conf.Configuration
[INFO]         new org.apache.hadoop.conf.Configuration."<init>" // def <init>(): org.apache.hadoop.conf.Configuration in class Configuration, tree.tpe=()org.apache.hadoop.conf.Configuration
[INFO]         Nil
[INFO]       )
[INFO]     )
[INFO]   )
[INFO]   DefDef( // def <init>(uri: String): com.example.core.compute.jobs.factstore.util.FileSystemFactory in class FileSystemFactory
[INFO]     <method> <triedcooking>
[INFO]     "<init>"
[INFO]     []
[INFO]     // 1 parameter list
[INFO]     ValDef( // uri: String
[INFO]       <param> <paramaccessor> <triedcooking>
[INFO]       "uri"
[INFO]       <tpt> // tree.tpe=String
[INFO]       <empty>
[INFO]     )
[INFO]     <tpt> // tree.tpe=com.example.core.compute.jobs.factstore.util.FileSystemFactory
[INFO]     Block( // tree.tpe=Unit
[INFO]       // 2 statements
[INFO]       Assign( // tree.tpe=Unit
[INFO]         FileSystemFactory.this."uri" // private[this] val uri: String in class FileSystemFactory, tree.tpe=String
[INFO]         "uri" // uri: String, tree.tpe=String
[INFO]       )
[INFO]       Apply( // def <init>(): Object in class Object, tree.tpe=Object
[INFO]         FileSystemFactory.super."<init>" // def <init>(): Object in class Object, tree.tpe=()Object
[INFO]         Nil
[INFO]       )
[INFO]       ()
[INFO]     )
[INFO]   )
[INFO] )
[INFO]
[INFO] == Expanded type of tree ==
[INFO]
[INFO] ConstantType(
[INFO]   value = Constant(com.example.core.compute.jobs.factstore.source.SourceDirectory)
[INFO]
[INFO] )
[INFO]
[INFO] uncaught exception during compilation: java.io.IOException
[ERROR] error: Operation not permitted
[WARNING] 7 warnings found
[ERROR] two errors found

However when I assign the for-comprehension to a val, it compiles without issue:

val result = for {
     _ <- markerUtil.uploadingStarted(source, destPath, files.map(_.path))
     inputStream <- createInputStream(files)
     _ <- store.upload(destPath, inputStream)
     _ <- markerUtil.uploadingDone(source)
     _ <- deleteFiles(files)
     _ <- markerUtil.uploadedDone(source)
   } yield UploadResult(inputSize, destPath)
   result

I also tried it by converting the for-yield to a flatMap, I get exactly the same errors, and it also gets resolved when I assign it to a val.

scabug commented 9 years ago

Imported From: https://issues.scala-lang.org/browse/SI-9132?orig=1 Reporter: Nathan Bijnens (nathan_gs) Affected Versions: 2.10.4

scabug commented 9 years ago

@retronym said: Is that SBT output? Can you please run last compile to show the full stack trace of the exception?

To hazard a guess, it sounds like a file system error writing a class file. Perhaps a file name of a class is too long for the underlying filesystem, or perhaps a file is on disk already with permissions that don't allow it to be overwritten.

scabug commented 9 years ago

@retronym said: (I've marked this as "incomplete" pending more details, please reopen as soon as you've got them and I'll take a further look)

scabug commented 9 years ago

Nathan Bijnens (nathan_gs) said: As indicated in the environment I am using maven 3.

I used this command to make sure I was starting 'fresh':

rm -rf target/ ; mvn clean

Then I tried to compile using following code:

mvn compile -e

Which gave following output:

[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Archive Factstore 1.4.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/maven-metadata.xml
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/maven-metadata.xml
Downloading: http://www.allanbank.com/repo/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/maven-metadata.xml

Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/core-common-datatypes/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/core-common-datatypes/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/core-common-datatypes/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://www.allanbank.com/repo/com/example/core-common-datatypes/1.0.0-SNAPSHOT/maven-metadata.xml

Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://www.allanbank.com/repo/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/maven-metadata.xml

Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: http://www.allanbank.com/repo/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml

Downloading: s3://example-artifacts/mvn/snapshot/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
[INFO] Logged in - example-artifacts
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
607 B   
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml.sha1

Downloaded: s3://example-artifacts/mvn/snapshot/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml (607 B at 0.1 KB/sec)
[INFO] Logged off - example-artifacts
[INFO] Transfers: 2 Time: 711ms Amount: 647 bytes Throughput: 909.986 bytes/s
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml

Downloading: http://www.allanbank.com/repo/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml

[INFO] Logged in - example-artifacts
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
788 B   
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml.sha1

Downloaded: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml (788 B at 0.4 KB/sec)
[INFO] Logged off - example-artifacts
[INFO] Transfers: 2 Time: 727ms Amount: 828 bytes Throughput: 1.112 KB/s
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml

Downloading: http://www.allanbank.com/repo/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml

[INFO] Logged in - example-artifacts
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
607 B   
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml.sha1

Downloaded: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml (607 B at 0.4 KB/sec)
[INFO] Logged off - example-artifacts
[INFO] Transfers: 2 Time: 674ms Amount: 647 bytes Throughput: 959.941 bytes/s
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml

Downloading: http://www.allanbank.com/repo/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml

[INFO] Logged in - example-artifacts
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
787 B   
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml.sha1

Downloaded: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml (787 B at 0.4 KB/sec)
[INFO] Logged off - example-artifacts
[INFO] Transfers: 2 Time: 637ms Amount: 827 bytes Throughput: 1.268 KB/s
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:parse-version (parse-version) @ core-compute-jobs-factstore ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.6.5.201403032054:prepare-agent (default-prepare-agent) @ core-compute-jobs-factstore ---
[INFO] argLine set to -javaagent:/home/vagrant/.m2/repository/org/jacoco/org.jacoco.agent/0.6.5.201403032054/org.jacoco.agent-0.6.5.201403032054-runtime.jar=destfile=/vagrant/core-compute-jobs-factstore/target/jacoco.exec
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:add-source (add-scala-sources) @ core-compute-jobs-factstore ---
[INFO] Source directory: /vagrant/core-compute-jobs-factstore/src/main/scala added.
[INFO] 
[INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ core-compute-jobs-factstore ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ core-compute-jobs-factstore ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-scala-plugin:2.15.2:compile (default) @ core-compute-jobs-factstore ---
[INFO] Checking for multiple versions of scala
Downloading: http://maven.twttr.com/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo.spray.io/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml

Downloading: http://repo.spray.io/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
Downloading: http://maven.twttr.com/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml

[WARNING]  Expected all dependencies to require Scala version: 2.10.4
[WARNING]  com.example:core-compute-jobs-factstore:1.4.0-SNAPSHOT requires scala version: 2.10.4
[WARNING]  com.twitter:chill_2.10:0.3.6 requires scala version: 2.10.3
[WARNING] Multiple versions of scala libraries detected!
[INFO] includes = [**/*.scala,**/*.java,]
[INFO] excludes = []
[INFO] /vagrant/core-compute-jobs-factstore/src/main/scala:-1: info: compiling
[INFO] Compiling 14 source files to /vagrant/core-compute-jobs-factstore/target/classes at 1422955019817
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[ERROR] error: 
[INFO]      while compiling: /vagrant/core-compute-jobs-factstore/src/main/scala/com/example/core/compute/jobs/factstore/util/FileSystemFactory.scala
[INFO]         during phase: jvm
[INFO]      library version: version 2.10.4
[INFO]     compiler version: version 2.10.4
[INFO]   reconstructed args: -classpath /vagrant/core-compute-jobs-factstore/target/classes:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-client/2.0.0-mr1-cdh4.4.0/hadoop-client-2.0.0-mr1-cdh4.4.0.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-common/2.0.0-cdh4.4.0/hadoop-common-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-annotations/2.0.0-cdh4.4.0/hadoop-annotations-2.0.0-cdh4.4.0.jar:/usr/lib/jvm/java-7-openjdk-amd64/jre/../lib/tools.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/home/vagrant/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/home/vagrant/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:/home/vagrant/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/home/vagrant/.m2/repository/commons-lang/commons-lang/2.5/commons-lang-2.5.jar:/home/vagrant/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/home/vagrant/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/home/vagrant/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/home/vagrant/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/home/vagrant/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/home/vagrant/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:/home/vagrant/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:/home/vagrant/.m2/repository/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5.jar:/home/vagrant/.m2/repository/org/apache/avro/avro/1.7.4/avro-1.7.4.jar:/home/vagrant/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/home/vagrant/.m2/repository/org/tukaani/xz/1.0/xz-1.0.jar:/home/vagrant/.m2/repository/com/google/protobuf/protobuf-java/2.4.0a/protobuf-java-2.4.0a.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-auth/2.0.0-cdh4.4.0/hadoop-auth-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/com/jcraft/jsch/0.1.42/jsch-0.1.42.jar:/home/vagrant/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5-cdh4.4.0/zookeeper-3.4.5-cdh4.4.0.jar:/home/vagrant/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.0.0-cdh4.4.0/hadoop-hdfs-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar:/home/vagrant/.m2/repository/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar:/home/vagrant/.m2/repository/asm/asm/3.1/asm-3.1.jar:/home/vagrant/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-core/2.0.0-mr1-cdh4.4.0/hadoop-core-2.0.0-mr1-cdh4.4.0.jar:/home/vagrant/.m2/repository/hsqldb/hsqldb/1.8.0.10/hsqldb-1.8.0.10.jar:/home/vagrant/.m2/repository/com/amazonaws/aws-java-sdk/1.6.4/aws-java-sdk-1.6.4.jar:/home/vagrant/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/home/vagrant/.m2/repository/commons-codec/commons-codec/1.3/commons-codec-1.3.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.1.1/jackson-core-2.1.1.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.1.1/jackson-databind-2.1.1.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.1.1/jackson-annotations-2.1.1.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar:/home/vagrant/.m2/repository/org/apache/spark/spark-core_2.10/1.1.1/spark-core_2.10-1.1.1.jar:/home/vagrant/.m2/repository/net/java/dev/jets3t/jets3t/0.7.1/jets3t-0.7.1.jar:/home/vagrant/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-recipes/2.4.0/curator-recipes-2.4.0.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-framework/2.4.0/curator-framework-2.4.0.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-client/2.4.0/curator-client-2.4.0.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-plus/8.1.14.v20131031/jetty-plus-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.transaction/1.1.1.v201105210645/javax.transaction-1.1.1.v201105210645.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-webapp/8.1.14.v20131031/jetty-webapp-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-xml/8.1.14.v20131031/jetty-xml-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-servlet/8.1.14.v20131031/jetty-servlet-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-jndi/8.1.14.v20131031/jetty-jndi-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.mail.glassfish/1.4.1.v201005082020/javax.mail.glassfish-1.4.1.v201005082020.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.activation/1.1.0.v201105071233/javax.activation-1.1.0.v201105071233.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-security/8.1.14.v20131031/jetty-security-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-util/8.1.14.v20131031/jetty-util-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-server/8.1.14.v20131031/jetty-server-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-continuation/8.1.14.v20131031/jetty-continuation-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-http/8.1.14.v20131031/jetty-http-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-io/8.1.14.v20131031/jetty-io-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/com/google/guava/guava/14.0.1/guava-14.0.1.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/home/vagrant/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/home/vagrant/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar:/home/vagrant/.m2/repository/org/slf4j/jul-to-slf4j/1.7.5/jul-to-slf4j-1.7.5.jar:/home/vagrant/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.5/jcl-over-slf4j-1.7.5.jar:/home/vagrant/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/home/vagrant/.m2/repository/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar:/home/vagrant/.m2/repository/com/ning/compress-lzf/1.0.0/compress-lzf-1.0.0.jar:/home/vagrant/.m2/repository/org/xerial/snappy/snappy-java/1.0.5.3/snappy-java-1.0.5.3.jar:/home/vagrant/.m2/repository/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar:/home/vagrant/.m2/repository/com/twitter/chill_2.10/0.3.6/chill_2.10-0.3.6.jar:/home/vagrant/.m2/repository/com/twitter/chill-java/0.3.6/chill-java-0.3.6.jar:/home/vagrant/.m2/repository/commons-net/commons-net/2.2/commons-net-2.2.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-remote_2.10/2.2.3-shaded-protobuf/akka-remote_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-actor_2.10/2.2.3-shaded-protobuf/akka-actor_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/io/netty/netty/3.6.6.Final/netty-3.6.6.Final.jar:/home/vagrant/.m2/repository/org/spark-project/protobuf/protobuf-java/2.4.1-shaded/protobuf-java-2.4.1-shaded.jar:/home/vagrant/.m2/repository/org/uncommons/maths/uncommons-maths/1.2.2a/uncommons-maths-1.2.2a.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-slf4j_2.10/2.2.3-shaded-protobuf/akka-slf4j_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/org/json4s/json4s-jackson_2.10/3.2.10/json4s-jackson_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/json4s/json4s-core_2.10/3.2.10/json4s-core_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/json4s/json4s-ast_2.10/3.2.10/json4s-ast_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/scala-lang/scalap/2.10.0/scalap-2.10.0.jar:/home/vagrant/.m2/repository/colt/colt/1.2.0/colt-1.2.0.jar:/home/vagrant/.m2/repository/concurrent/concurrent/1.3.4/concurrent-1.3.4.jar:/home/vagrant/.m2/repository/org/apache/mesos/mesos/0.18.1/mesos-0.18.1-shaded-protobuf.jar:/home/vagrant/.m2/repository/io/netty/netty-all/4.0.23.Final/netty-all-4.0.23.Final.jar:/home/vagrant/.m2/repository/com/clearspring/analytics/stream/2.7.0/stream-2.7.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-core/3.0.0/metrics-core-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-jvm/3.0.0/metrics-jvm-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-json/3.0.0/metrics-json-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-graphite/3.0.0/metrics-graphite-3.0.0.jar:/home/vagrant/.m2/repository/org/tachyonproject/tachyon-client/0.5.0/tachyon-client-0.5.0.jar:/home/vagrant/.m2/repository/org/tachyonproject/tachyon/0.5.0/tachyon-0.5.0.jar:/home/vagrant/.m2/repository/org/spark-project/pyrolite/2.0.1/pyrolite-2.0.1.jar:/home/vagrant/.m2/repository/net/sf/py4j/py4j/0.8.2.1/py4j-0.8.2.1.jar:/home/vagrant/.m2/repository/joda-time/joda-time/2.2/joda-time-2.2.jar:/home/vagrant/.m2/repository/org/apache/httpcomponents/httpclient/4.2.6/httpclient-4.2.6.jar:/home/vagrant/.m2/repository/org/apache/httpcomponents/httpcore/4.2.5/httpcore-4.2.5.jar:/home/vagrant/.m2/repository/com/esotericsoftware/kryo/kryo/2.22/kryo-2.22.jar:/home/vagrant/.m2/repository/com/typesafe/config/1.2.1/config-1.2.1.jar:/home/vagrant/.m2/repository/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/core-common-datatypes-domain-1.6.0-SNAPSHOT.jar:/home/vagrant/.m2/repository/org/apache/thrift/libthrift/0.9.1/libthrift-0.9.1.jar:/home/vagrant/.m2/repository/com/twitter/scrooge-core_2.10/3.12.3/scrooge-core_2.10-3.12.3.jar:/home/vagrant/.m2/repository/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/core-compute-util-spark-metrics-1.0.0-SNAPSHOT.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-ganglia/3.0.1/metrics-ganglia-3.0.1.jar:/home/vagrant/.m2/repository/info/ganglia/gmetric4j/gmetric4j/1.0.3/gmetric4j-1.0.3.jar:/home/vagrant/.m2/repository/org/acplt/oncrpc/1.0.7/oncrpc-1.0.7.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-reflect/2.10.3/scala-reflect-2.10.3.jar:/home/vagrant/.m2/repository/junit/junit/4.11/junit-4.11.jar:/home/vagrant/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-compiler/2.10.3/scala-compiler-2.10.3.jar -d /vagrant/core-compute-jobs-factstore/target/classes
[INFO] 
[INFO]   last tree to typer: Literal(Constant(com.example.core.compute.jobs.factstore.source.SourceDirectory))
[INFO]               symbol: null
[INFO]    symbol definition: null
[INFO]                  tpe: Class(classOf[com.example.core.compute.jobs.factstore.source.SourceDirectory])
[INFO]        symbol owners: 
[INFO]       context owners: class FileSystemFactory -> package util
[INFO] 
[INFO] == Enclosing template or block ==
[INFO] 
[INFO] Template( // val <local FileSystemFactory>: <notype> in class FileSystemFactory, tree.tpe=com.example.core.compute.jobs.factstore.util.FileSystemFactory
[INFO]   "java.lang.Object", "scala.Serializable" // parents
[INFO]   ValDef(
[INFO]     private
[INFO]     "_"
[INFO]     <tpt>
[INFO]     <empty>
[INFO]   )
[INFO]   // 3 statements
[INFO]   ValDef( // private[this] val uri: String in class FileSystemFactory
[INFO]     private <local> <paramaccessor> <triedcooking>
[INFO]     "uri"
[INFO]     <tpt> // tree.tpe=String
[INFO]     <empty>
[INFO]   )
[INFO]   DefDef( // def get(): org.apache.hadoop.fs.FileSystem in class FileSystemFactory
[INFO]     <method> <triedcooking>
[INFO]     "get"
[INFO]     []
[INFO]     List(Nil)
[INFO]     <tpt> // tree.tpe=org.apache.hadoop.fs.FileSystem
[INFO]     Apply( // def get(x$1: java.net.URI,x$2: org.apache.hadoop.conf.Configuration): org.apache.hadoop.fs.FileSystem in object FileSystem, tree.tpe=org.apache.hadoop.fs.FileSystem
[INFO]       "org"."apache"."hadoop"."fs"."HFileSystem"."get" // def get(x$1: java.net.URI,x$2: org.apache.hadoop.conf.Configuration): org.apache.hadoop.fs.FileSystem in object FileSystem, tree.tpe=(x$1: java.net.URI, x$2: org.apache.hadoop.conf.Configuration)org.apache.hadoop.fs.FileSystem
[INFO]       // 2 arguments
[INFO]       Apply( // def <init>(x$1: String): java.net.URI in class URI, tree.tpe=java.net.URI
[INFO]         new java.net.URI."<init>" // def <init>(x$1: String): java.net.URI in class URI, tree.tpe=(x$1: String)java.net.URI
[INFO]         FileSystemFactory.this."uri" // private[this] val uri: String in class FileSystemFactory, tree.tpe=String
[INFO]       )
[INFO]       Apply( // def <init>(): org.apache.hadoop.conf.Configuration in class Configuration, tree.tpe=org.apache.hadoop.conf.Configuration
[INFO]         new org.apache.hadoop.conf.Configuration."<init>" // def <init>(): org.apache.hadoop.conf.Configuration in class Configuration, tree.tpe=()org.apache.hadoop.conf.Configuration
[INFO]         Nil
[INFO]       )
[INFO]     )
[INFO]   )
[INFO]   DefDef( // def <init>(uri: String): com.example.core.compute.jobs.factstore.util.FileSystemFactory in class FileSystemFactory
[INFO]     <method> <triedcooking>
[INFO]     "<init>"
[INFO]     []
[INFO]     // 1 parameter list
[INFO]     ValDef( // uri: String
[INFO]       <param> <paramaccessor> <triedcooking>
[INFO]       "uri"
[INFO]       <tpt> // tree.tpe=String
[INFO]       <empty>
[INFO]     )
[INFO]     <tpt> // tree.tpe=com.example.core.compute.jobs.factstore.util.FileSystemFactory
[INFO]     Block( // tree.tpe=Unit
[INFO]       // 2 statements
[INFO]       Assign( // tree.tpe=Unit
[INFO]         FileSystemFactory.this."uri" // private[this] val uri: String in class FileSystemFactory, tree.tpe=String
[INFO]         "uri" // uri: String, tree.tpe=String
[INFO]       )
[INFO]       Apply( // def <init>(): Object in class Object, tree.tpe=Object
[INFO]         FileSystemFactory.super."<init>" // def <init>(): Object in class Object, tree.tpe=()Object
[INFO]         Nil
[INFO]       )
[INFO]       ()
[INFO]     )
[INFO]   )
[INFO] )
[INFO] 
[INFO] == Expanded type of tree ==
[INFO] 
[INFO] ConstantType(
[INFO]   value = Constant(com.example.core.compute.jobs.factstore.source.SourceDirectory)
[INFO]   
[INFO] )
[INFO] 
[INFO] uncaught exception during compilation: java.io.IOException
[ERROR] error: Operation not permitted
[WARNING] 7 warnings found
[ERROR] two errors found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:11.629s
[INFO] Finished at: Tue Feb 03 09:18:55 UTC 2015
[INFO] Final Memory: 20M/50M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project core-compute-jobs-factstore: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project core-compute-jobs-factstore: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
    at org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:350)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
    ... 19 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
    at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:346)
    at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:149)
    at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:136)
    at org_scala_tools_maven_executions.JavaMainCallerByFork.run(JavaMainCallerByFork.java:80)
    at org_scala_tools_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:124)
    at org_scala_tools_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:80)
    at org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:342)
    ... 21 more
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

If I assign the for-comprehension to a value, repeat all the steps (rm target & clean, mvn compile) it compiles without error. I am very sure it's not a permission or disk issue.

[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Archive Factstore 1.4.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:parse-version (parse-version) @ core-compute-jobs-factstore ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.6.5.201403032054:prepare-agent (default-prepare-agent) @ core-compute-jobs-factstore ---
[INFO] argLine set to -javaagent:/home/vagrant/.m2/repository/org/jacoco/org.jacoco.agent/0.6.5.201403032054/org.jacoco.agent-0.6.5.201403032054-runtime.jar=destfile=/vagrant/core-compute-jobs-factstore/target/jacoco.exec
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:add-source (add-scala-sources) @ core-compute-jobs-factstore ---
[INFO] Source directory: /vagrant/core-compute-jobs-factstore/src/main/scala added.
[INFO] 
[INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ core-compute-jobs-factstore ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ core-compute-jobs-factstore ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-scala-plugin:2.15.2:compile (default) @ core-compute-jobs-factstore ---
[INFO] Checking for multiple versions of scala
[WARNING]  Expected all dependencies to require Scala version: 2.10.4
[WARNING]  com.example:core-compute-jobs-factstore:1.4.0-SNAPSHOT requires scala version: 2.10.4
[WARNING]  com.twitter:chill_2.10:0.3.6 requires scala version: 2.10.3
[WARNING] Multiple versions of scala libraries detected!
[INFO] includes = [**/*.scala,**/*.java,]
[INFO] excludes = []
[INFO] /vagrant/core-compute-jobs-factstore/src/main/scala:-1: info: compiling
[INFO] Compiling 14 source files to /vagrant/core-compute-jobs-factstore/target/classes at 1422955813091
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: there were 2 deprecation warning(s); re-run with -deprecation for details
[WARNING] 8 warnings found
[INFO] prepare-compile in 0 s
[INFO] compile in 148 s
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:18.537s
[INFO] Finished at: Tue Feb 03 09:32:41 UTC 2015
[INFO] Final Memory: 16M/39M
[INFO] ------------------------------------------------------------------------
scabug commented 9 years ago

@retronym said: Sorry, I didn't spot that you'd already specified Mavan in the bug report. Unfortunately it seems to be swallowing the interesting part of the exception stack trace (in a similar manner to the default behaviour of SBT!)

Is your project in an encrypted home directory in Ubuntu? The long class names generated by heavily nested closures (as are generated in for comprehensions with many generators) can exceed the maximum file path allowed. See for example https://github.com/scala/pickling/issues/10 / http://brianoneill.blogspot.com.au/2012/10/file-name-too-long-in-scalasbt-on-ubuntu.html

scabug commented 9 years ago

Nathan Bijnens (nathan_gs) said: I am not using any encryption. I don't think it has something to do with the length of the file paths.

I find it very strange that this doesn't work:

for {
     _ <- markerUtil.uploadingStarted(source, destPath, files.map(_.path)) // Try[Unit]
     inputStream <- createInputStream(files) // Try[InputStream]
     _ <- store.upload(destPath, inputStream) // Try[Unit]
     _ <- markerUtil.uploadingDone(source) // Try[Unit]
     _ <- deleteFiles(files) // Try[Unit]
     _ <- markerUtil.uploadedDone(source) // Try[Unit]
   } yield UploadResult(inputSize, destPath)

while this works:

val result = for {
     _ <- markerUtil.uploadingStarted(source, destPath, files.map(_.path))
     inputStream <- createInputStream(files)
     _ <- store.upload(destPath, inputStream)
     _ <- markerUtil.uploadingDone(source)
     _ <- deleteFiles(files)
     _ <- markerUtil.uploadedDone(source)
   } yield UploadResult(inputSize, destPath)
result
scabug commented 9 years ago

@retronym said (edited on Feb 3, 2015 12:50:59 PM UTC): The two snippets subtly change the way that anonymous classes are named; they end up including the method name as a prefix of the mangled name in one case but not in the other.

% cat sandbox/test.scala
object Test {
  def test = {
    val result = for {
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
     } yield ()
   }
}

% scalac-hash v2.11.5 -d /tmp/classes sandbox/test.scala

% find /tmp/classes
/tmp/classes/
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3$$anonfun$apply$4$$anonfun$apply$5.class
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3$$anonfun$apply$4.class
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3.class
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2.class
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1.class
/tmp/classes/Test$$anonfun$1.class
/tmp/classes/Test$.class
/tmp/classes/Test$markerUtil$.class
/tmp/classes/Test$store$.class
/tmp/classes/Test$UploadResult$.class
/tmp/classes/Test$UploadResult.class
/tmp/classes/Test.class

% rm -rf /tmp/classes/*

% subl sandbox/test.scala

% cat sandbox/test.scala
object Test {
  def test = {
    for {
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
     } yield ()
   }
}

% scalac-hash v2.11.5 -d /tmp/classes sandbox/test.scala

% find /tmp/classes
/tmp/classes
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3$$anonfun$apply$4$$anonfun$apply$5.class
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3$$anonfun$apply$4.class
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3.class
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1$$anonfun$apply$2.class
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1.class
/tmp/classes/Test$$anonfun$test$1.class
/tmp/classes/Test$.class
/tmp/classes/Test$markerUtil$.class
/tmp/classes/Test$store$.class
/tmp/classes/Test$UploadResult$.class
/tmp/classes/Test$UploadResult.class
/tmp/classes/Test.class

Could I ask you to try to compile this program on your system using the command line version of the Scala compiler?

object Test {
  def test = {
    val r = for {
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
     } yield ()
   }
}
scabug commented 9 years ago

@retronym said: Could you also please double check for encrypted home partition with these instructions? http://askubuntu.com/questions/53242/check-if-partition-is-encrypted

scabug commented 9 years ago

Nathan Bijnens (nathan_gs) said: It was on a Vagrant/Virtualbox machine, with a Windows host, apparently there is a path length limit, which gives strange errors like these. This issue can be closed, although the error messages could be a little more cleaner (for instance, a reference to the path length).

som-snytt commented 6 years ago

@retronym This was a nice one to read.