TiarkRompf / virtualization-lms-core

A Framework for Runtime Code Generation and Compiled DSLs
http://scala-lms.github.com
BSD 3-Clause "New" or "Revised" License
324 stars 91 forks source link

ClassCastException when using effects with ArrayBuffer #93

Closed biboudis closed 9 years ago

biboudis commented 9 years ago

When trying to emit (dynamically compile) the following snippet/test (static compile passes)

trait Tests extends IfThenElse
    with BooleanOps
    with Variables
    with OrderingOps
    with PrimitiveOps
    with LiftVariables
    with LiftBoolean 
    with LiftPrimitives 
    with ArrayOps 
    with ArrayBufferOps
    with While 
    with NumericOps
    with Equal {
  def toArrayTest (xs : Rep[Array[Int]]) : Rep[Array[Int]] = { 
    val buffer = unit(new ArrayBuffer[Int]())
    buffer += xs(0)
    buffer += xs(1)
    buffer.toArray
  }
}

with the following configuration:

  val staged = new Tests
      with PrimitiveOpsExp 
      with ArrayOpsExp 
      with IfThenElseExp
      with BooleanOpsExp
      with OrderingOpsExp
      with NumericOpsExp
      with WhileExpOptSpeculative  
      with VariablesExpOpt
      with EqualExpOpt
      with ArrayBufferOpsExp
      with ScalaCompile { self =>
    val codegen = new ScalaGenEffect 
        with ScalaGenPrimitiveOps 
        with ScalaGenArrayOps
        with ScalaGenIfThenElse
        with ScalaGenBooleanOps
        with ScalaGenOrderingOps
        with ScalaGenWhileOptSpeculative 
        with ScalaGenNumericOps
        with ScalaGenEqual
        with ScalaGenArrayBufferOps
        with ScalaGenVariables { val IR: self.type = self }

    codegen.emitSource(self.toArrayTest, "toArrayTest", new PrintWriter(System.out))
    codegen.reset
    val toArrayTest : (Array[Int] => Array[Int]) = compile(self.toArrayTest)

I get this exception: java.lang.ClassCastException: scala.virtualization.lms.internal.Expressions$Const cannot be cast to scala.virtualization.lms.internal.Expressions$Sym

Is there something that I can fix to avoid this behavior? The full log follows.

> last test:test
[debug] Running TaskDef(MySpec, sbt.SubclassFingerprintWrapper@b338f86, false, [SuiteSelector])
java.lang.ExceptionInInitializerError
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.scalacheck.ScalaCheckFramework$$anon$2.run(ScalaCheckFramework.scala:83)
    at sbt.RunnerWrapper$1.runRunner2(FrameworkWrapper.java:223)
    at sbt.RunnerWrapper$1.execute(FrameworkWrapper.java:236)
    at sbt.TestRunner.runTest$1(TestFramework.scala:76)
    at sbt.TestRunner.run(TestFramework.scala:85)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
    at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
    at sbt.TestFunction.apply(TestFramework.scala:207)
    at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
    at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
    at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
    at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassCastException: scala.virtualization.lms.internal.Expressions$Const cannot be cast to scala.virtualization.lms.internal.Expressions$Sym
    at scala.virtualization.lms.internal.Effects$$anonfun$reflectEffectInternal$5.apply(Effects.scala:457)
    at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:778)
    at scala.collection.immutable.List.foreach(List.scala:381)
    at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:777)
    at scala.virtualization.lms.internal.Effects$class.reflectEffectInternal(Effects.scala:457)
    at MySpec$$anon$1.reflectEffectInternal(MySpec.scala:31)
    at scala.virtualization.lms.internal.Effects$class.reflectEffect(Effects.scala:430)
    at MySpec$$anon$1.reflectEffect(MySpec.scala:31)
    at scala.virtualization.lms.internal.Effects$class.reflectWrite(Effects.scala:418)
    at MySpec$$anon$1.reflectWrite(MySpec.scala:31)
    at scala.virtualization.lms.common.ArrayBufferOpsExp$class.arraybuffer_append(ArrayBufferOps.scala:59)
    at MySpec$$anon$1.arraybuffer_append(MySpec.scala:31)
    at MySpec$$anon$1.arraybuffer_append(MySpec.scala:31)
    at scala.virtualization.lms.common.ArrayBufferOps$class.infix_$plus$eq(ArrayBufferOps.scala:28)
    at MySpec$$anon$1.infix_$plus$eq(MySpec.scala:31)
    at Tests$class.toArrayTest(MySpec.scala:18)
    at MySpec$$anon$1.toArrayTest(MySpec.scala:31)
    at MySpec$$anon$1$$anonfun$5.apply(MySpec.scala:62)
    at MySpec$$anon$1$$anonfun$5.apply(MySpec.scala:62)
    at scala.virtualization.lms.internal.GenericCodegen$$anonfun$1.apply(GenericCodegen.scala:114)
    at scala.virtualization.lms.internal.GenericCodegen$$anonfun$1.apply(GenericCodegen.scala:114)
    at scala.virtualization.lms.internal.Expressions$class.reifySubGraph(Expressions.scala:86)
    at MySpec$$anon$1.reifySubGraph(MySpec.scala:31)
    at scala.virtualization.lms.internal.Effects$class.reifyEffects(Effects.scala:572)
    at MySpec$$anon$1.reifyEffects(MySpec.scala:31)
    at scala.virtualization.lms.internal.NestedBlockTraversal$class.reifyBlock(BlockTraversal.scala:41)
    at MySpec$$anon$1$$anon$2.reifyBlock(MySpec.scala:43)
    at MySpec$$anon$1$$anon$2.reifyBlock(MySpec.scala:43)
    at scala.virtualization.lms.internal.GenericCodegen$class.emitSource(GenericCodegen.scala:114)
    at MySpec$$anon$1$$anon$2.emitSource(MySpec.scala:43)
    at MySpec$$anon$1.<init>(MySpec.scala:62)
    at MySpec$.<init>(MySpec.scala:31)
    at MySpec$.<clinit>(MySpec.scala)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.scalacheck.ScalaCheckFramework$$anon$2.run(ScalaCheckFramework.scala:83)
    at sbt.RunnerWrapper$1.runRunner2(FrameworkWrapper.java:223)
    at sbt.RunnerWrapper$1.execute(FrameworkWrapper.java:236)
    at sbt.TestRunner.runTest$1(TestFramework.scala:76)
    at sbt.TestRunner.run(TestFramework.scala:85)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
    at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
    at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
    at sbt.TestFunction.apply(TestFramework.scala:207)
    at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
    at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
    at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
    at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

ping @TiarkRompf

astojanov commented 9 years ago

This is because calling

unit(new ArrayBuffer[Int]())

does not crate an IR Def which corresponds to creation of a new array buffer, instead it creates a const of the Scala ArrayBuffer that is created by new. To avoid this, try:

def toArrayTest (xs : Rep[Array[Int]]) : Rep[Array[Int]] = {
  val buffer = ArrayBuffer[Int]()
  buffer += xs(0)
  buffer += xs(1)
  buffer.toArray
}
biboudis commented 9 years ago

Indeed. Thank you!

Regarding the following, I managed to avoid it by implementing the toArray method (in my streaming library) by an explicit loop function instead of foldingLeft (foldLeft(buf)((b, v) => { b += v; b })), but what is the general advise here? Avoid aliasing of mutable references in general? BTW, the code compiles and runs, despite the illegal sharing error.

error: illegal sharing of mutable objects Sym(64)
at Sym(65)=Reflect(NewVar(Sym(64)),Summary(false,false,false,false,true,false,List(Sym(64)),List(Sym(64)),List(),List()),List(Sym(64)))
/*****************************************
  Emitting Generated Code                  
*******************************************/
class toArrayTest extends ((Array[Int])=>(Array[Int])) {
def apply(x63:Array[Int]): Array[Int] = {
val x64 = scala.collection.mutable.ArrayBuffer[Int]()
var x65: scala.collection.mutable.ArrayBuffer[Int] = x64
var x66: Int = 0
var x67: Boolean = true
val x68 = x63.length
val x100 = while ({val x90 = x66
val x91 = x90 < x68
val x92 = x91 && true
x92}) {
val x95 = x66
val x96 = x63(x95)
val x97 = x64 += x96
val x98 = x66 += 1
()
}
val x101 = x64.toArray
x101
}
}
/*****************************************
  End of Generated Code                  
*******************************************/
error: illegal sharing of mutable objects Sym(1)
at Sym(2)=Reflect(NewVar(Sym(1)),Summary(false,false,false,false,true,false,List(Sym(1)),List(Sym(1)),List(),List()),List(Sym(1)))

Thank you @astojanov

TiarkRompf commented 9 years ago

Yes, the general advice is to avoid sharing and nesting of mutable objects. So instead of putting the ArrayBuffer into a var, it is better to bind it using a val, and update it directly.

TiarkRompf commented 9 years ago

Btw, in your original snippet you were using unit(new ArrayBuffer()). Semantically, this would mean that you are creating an ArrayBuffer at code generation time, and then having it filled when the generated code is run. Sometimes this is indeed useful, but you'd have to use staticData instead of unit, because unit only works with primitives, not heap-allocated objects.

biboudis commented 9 years ago

Thank you for the tips @TiarkRompf and @astojanov!!! Tiark, especially your second comment made the semantics of unit more clear to me!