search

CANVE / extractor

Extracts and normalizes the type relationships and call graph of scala sbt projects.
4 stars 1 forks source link

Inject appropriate binary version of the compiler plugin per sub-project #12

Open matanox opened 8 years ago

matanox commented 8 years ago

Running Canve for play framework (need to cd to /framework first), for a certain subproject, the classpath is clearly bringing in scala 2.10, whereas the resolved dependency of the compiler plugin is 2.11:

[canve] instrumenting project SBT-Run-Support...
[info] Compiling 12 Scala sources to /repos/canve/extractor/sbt-plugin-integration-test/target/scala-2.11/classes/integration-test-projects/playframework/framework/src/routes-compiler/target/sbt-routes-compiler/scala-2.10/classes...
[canve] instrumenting project SBT-Routes-Compiler...
[canve] extraction starting for project SBT-Run-Support (10 compilation units)
[canve] Settings {
  -classpath = /repos/canve/extractor/sbt-plugin-integration-test/target/scala-2.11/classes/integration-test-projects/playframework/framework/src/run-support/target/sbt-run-support/scala-2.10/classes:/repos/canve/extractor/sbt-plugin-integration-test/target/scala-2.11/classes/integration-test-projects/playframework/framework/src/build-link/target/classes:/repos/canve/extractor/sbt-plugin-integration-test/target/scala-2.11/classes/integration-test-projects/playframework/framework/src/play-exceptions/target/classes:/home/matan/.ivy2/cache/org.javassist/javassist/bundles/javassist-3.20.0-GA.jar:/home/matan/.ivy2/local/canve/compiler-plugin_2.10/0.0.1/jars/compiler-plugin_2.10.jar:/home/matan/.ivy2/local/canve/simple-graph_2.10/0.0.1/jars/simple-graph_2.10.jar:/home/matan/.ivy2/local/com.github.tototoshi/scala-csv_2.10/1.3.0-SNAPSHOT/jars/scala-csv_2.10.jar:/home/matan/.ivy2/cache/org.scala-sbt/io/jars/io-0.13.9.jar:/home/matan/.ivy2/cache/org.scala-sbt/control/jars/control-0.13.9.jar
  -P = List(canve:projectName:SBT-Run-Support)
  -d = .
  -deprecation = true
  -feature = true
  -encoding = utf8
  -unchecked = true
  -bootclasspath = /usr/lib/jvm/java-8-oracle/jre/lib/resources.jar:/usr/lib/jvm/java-8-oracle/jre/lib/rt.jar:/usr/lib/jvm/java-8-oracle/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jsse.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jce.jar:/usr/lib/jvm/java-8-oracle/jre/lib/charsets.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jfr.jar:/usr/lib/jvm/java-8-oracle/jre/classes:/home/matan/.sbt/boot/scala-2.10.5/lib/scala-library.jar
  -Xplugin = List(/home/matan/.ivy2/local/canve/compiler-plugin_2.11/0.0.1/jars/compiler-plugin_2.11.jar)
}

[canve] examining source file/repos/canve/extractor/sbt-plugin-integration-test/target/scala-2.11/classes/integration-test-projects/playframework/framework/src/run-support/src/main/scala/play/runsupport/RunHook.scala...
Could not determine source definition for symbol RunHook (7744) because start=end (361)
RunHook {

  /**
   * Called b
[error] 
[error]      while compiling: /repos/canve/extractor/sbt-plugin-integration-test/target/scala-2.11/classes/integration-test-projects/playframework/framework/src/run-support/src/main/scala/play/runsupport/RunHook.scala
[error]         during phase: canve
[error]      library version: version 2.10.5
[error]     compiler version: version 2.10.5
[error]   reconstructed args: -bootclasspath /usr/lib/jvm/java-8-oracle/jre/lib/resources.jar:/usr/lib/jvm/java-8-oracle/jre/lib/rt.jar:/usr/lib/jvm/java-8-oracle/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jsse.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jce.jar:/usr/lib/jvm/java-8-oracle/jre/lib/charsets.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jfr.jar:/usr/lib/jvm/java-8-oracle/jre/classes:/home/matan/.sbt/boot/scala-2.10.5/lib/scala-library.jar -P:canve:projectName:SBT-Run-Support -encoding utf8 -deprecation -feature -classpath /repos/canve/extractor/sbt-plugin-integration-test/target/scala-2.11/classes/integration-test-projects/playframework/framework/src/run-support/target/sbt-run-support/scala-2.10/classes:/repos/canve/extractor/sbt-plugin-integration-test/target/scala-2.11/classes/integration-test-projects/playframework/framework/src/build-link/target/classes:/repos/canve/extractor/sbt-plugin-integration-test/target/scala-2.11/classes/integration-test-projects/playframework/framework/src/play-exceptions/target/classes:/home/matan/.ivy2/cache/org.javassist/javassist/bundles/javassist-3.20.0-GA.jar:/home/matan/.ivy2/local/canve/compiler-plugin_2.10/0.0.1/jars/compiler-plugin_2.10.jar:/home/matan/.ivy2/local/canve/simple-graph_2.10/0.0.1/jars/simple-graph_2.10.jar:/home/matan/.ivy2/local/com.github.tototoshi/scala-csv_2.10/1.3.0-SNAPSHOT/jars/scala-csv_2.10.jar:/home/matan/.ivy2/cache/org.scala-sbt/io/jars/io-0.13.9.jar:/home/matan/.ivy2/cache/org.scala-sbt/control/jars/control-0.13.9.jar -Xplugin:/home/matan/.ivy2/local/canve/compiler-plugin_2.11/0.0.1/jars/compiler-plugin_2.11.jar -unchecked

This means runtime binary incompatibility, running the scala 2.11 plugin as part of a 2.10 compilation, which ultimately breaks on:

[error] uncaught exception during compilation: java.lang.NoSuchMethodError
java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
    at org.canve.compilerPlugin.ExtractedCodes.maybeAdd(ExtractedModel.scala:99)
    at org.canve.compilerPlugin.ExtractedModel.add(ExtractedModel.scala:54)
    at org.canve.compilerPlugin.TraversalExtraction$ExtractionTraversal$1.traverse(TraversalExtraction.scala:119)
    at org.canve.compilerPlugin.TraversalExtraction$ExtractionTraversal$1.traverse(TraversalExtraction.scala:19)
matanox commented 8 years ago

Looks like I'm adding the library dependency only once and not per project. Which brings in a single binary version of the compiler plugin - rather than fitting each sub-project with its matching binary version of the compiler plugin.

matanox commented 8 years ago

The workaround of the last commit here - which skips sub-projects that do not match the overall scala version - leaves cross-version support rather impaired, it only avoids crashing. Take parboiled2 as an example; there is no root project there, rather it uses the Common Settings idiom. So the project defaults to 2.10 while each and every sub-project is 2.11. We instrument nothing therefore there, through the sbt plugin.

matanox commented 8 years ago

libraryDependencies for sub-projects seem to have a management cycle of their own (e.g. seem to be ignored)

matanox commented 8 years ago

A different way around it - https://gitter.im/sbt/sbt?at=566d2350cffd648a05553745

@pfn I guess that would work if I had a proper sbt project to use for scoping, but I don’t. What I ended up doing is creating a configuration, scoping the ModuleID to the created configuration, and then use update in <my-configuration>.value.allFiles. That seems to give me all transitive dependencies. If I use instead dependencyClasspath in <my-configuration> I get back an error References to undefined settings: foo/<my-configuration>:dependencyClasspath from foo/*:<my-task>

matanox commented 8 years ago

This is going on hold now, until an sbt committer is willing to take this up one day or answer questions. It is just too cryptic and better support is needed. Noting that I suspect this may also be an issue for sub-projects which override their root project's library settings, not just to builds of mixed scala versions.

This issue is a hindrance to our breadth of support.