Cannot determine simple type name "repl"

[vejeta]

Mdoc versions affected 2.0.3/2.1.0 Sbt version 1.3.4

After migrating from tut to mdoc I found this error.

The snippets are like this:

val apartments: Seq[Apartment] = Seq(
  Apartment("Paris", 50,  300000.0, 2),
  Apartment("Paris", 100, 450000.0, 3),
  Apartment("Paris", 25,  250000.0, 1),
  Apartment("Lyon",  83,  200000.0, 2),
  Apartment("Lyon",  45,  133000.0, 1),
  Apartment("Nice",  74,  325000.0, 3)
)

val aptTypedDs0 = TypedDataset.create(apartments)

When running sbt docs/mdoc:

error: /development/frameless/./docs/src/main/tut/FeatureOverview.md:49:1: Error while encoding: java.util.concurrent.ExecutionException: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 41, Column 6: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 41, Column 6: Cannot determine simple type name "repl"
staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, input[0, repl.Session$App$Apartment, false].city, true, true) AS city#0
input[0, repl.Session$App$Apartment, false].surface AS surface#1
input[0, repl.Session$App$Apartment, false].price AS price#2
input[0, repl.Session$App$Apartment, false].bedrooms AS bedrooms#3
val aptTypedDs0 = TypedDataset.create(apartments)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The full stack trace is attached in this file:

mdoc-frameless-error.log

You can see the source .md here: https://github.com/vejeta/frameless/blob/update-dependencies/docs/src/main/tut/FeatureOverview.md

[olafurpg]

Thank you for reporting! What happens if you wrap the code fence in mdoc:reset-object?

[olafurpg]

Also, can you include the output from --verbose? It should include the generated source code that we feed to the compiler

[vejeta]

Thanks for the quick reply, @olafurpg!

If I do add scala mdoc:reset-object to that code fence, it does not find the types imported on a previous code fence. If I add it on the first code fence (the one with the imports), it gives something different:

Error while encoding: java.util.concurrent.ExecutionException: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 41, Column 6: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 41, Column 6: Cannot determine simple type name "repl"
staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, input[0, repl.Session$App0$Apartment, false].city, true, true) AS city#0
input[0, repl.Session$App0$Apartment, false].surface AS surface#1
input[0, repl.Session$App0$Apartment, false].price AS price#2
input[0, repl.Session$App0$Apartment, false].bedrooms AS bedrooms#3
val aptTypedDs0 = TypedDataset.create(apartments)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
java.lang.ExceptionInInitializerError
        at repl.Session$App.<init>(FeatureOverview.md:5)
        at repl.Session$.app(FeatureOverview.md:3)

[vejeta]

Here goes the instrumented code: instrumented_code.txt

[olafurpg]

Thank you for the information. Could you try to minimize the markdown document? For example remove the fail blocks. At a quick glance, this looks like an issue with the macro and not mdoc. The markdown file is translated to the source tile printed by —verbose and then compiled with the Scala compiler.

It could be a hygiene issue in the macro, for example it might be referencing an unqualified name “repl” that binds to the package produced by mdoc.

[olafurpg]

You could try to see it changing the package name in mdoc to something else than “repl” fixes the issue. You can clone the mdoc repo and update Instrumenter.scala and publish a local version.

[vejeta]

Aha! I followed that approach, I published it locally and used it after changing the package name, but got the same errors that I get while running the tests on the mdoc repo.

the compiler produced no classfiles and reported no errors to explain what went wrong during compilation.

e.g.:

sbt:mdocRoot> test
[info] RelativizeSuite:
[info] - a:href is processed
[info] - img:src is processed
[info] - script:src is processed
[info] - link:href is processed
[info] - cross-page
[info] - fragment is preserved
[info] - // is forced to https
[info] - directory/ is expanded to directory/index.html
error: eval.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: compile-only.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: await.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: mismatch.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: one.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: trailing-comment.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: triplequote.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
    """|error: timeout.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: reuse.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: last-statement.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: two.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
error: overload.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.
[info] CompileOnlySuite:
[info] - compile-only *** FAILED ***
[info]   reporter.hasErrors was true error: compile-only.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues. (BaseMarkdownSuite.scala:96)
[info] - reuse *** FAILED ***
[info]   reporter.hasErrors was true error: reuse.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues. (BaseMarkdownSuite.scala:96)

Is there any prerequisite to build mdoc that I am missing?

[olafurpg]

Can you push your diff? The "repl.Session" string is referenced somewhere else when evaluating the code

[vejeta]

Diff in Instrumenter code:

--- a/mdoc/src/main/scala/mdoc/internal/markdown/Instrumenter.scala
+++ b/mdoc/src/main/scala/mdoc/internal/markdown/Instrumenter.scala
@@ -109,7 +109,7 @@ object Instrumenter {

   def wrapBody(body: String): String = {
     val wrapped = new StringBuilder()
-      .append("package repl\n")
+      .append("package repl_instrumenter\n")
       .append("object Session extends _root_.mdoc.internal.document.DocumentBuilder {\n")
       .append("  def app(): _root_.scala.Unit = {val _ = new App()}\n")
       .append("  class App {\n")

Result when issuing docs/mdoc --verbose:

package repl_instrumenter
object Session extends _root_.mdoc.internal.document.DocumentBuilder {
  def app(): _root_.scala.Unit = {val _ = new App()}
  class App {
App0
}
object App0 {

$doc.startSection();
$doc.startStatement(0, 0, 0, 33);
import org.apache.spark.SparkConf
$doc.endStatement();
$doc.startStatement(1, 0, 1, 40);
import org.apache.spark.sql.SparkSession
$doc.endStatement();
$doc.startStatement(2, 0, 2, 29);
import frameless.TypedDataset
$doc.endStatement();
$doc.startStatement(4, 0, 4, 78);
case class Apartment(city: String, surface: Int, price: Double, bedrooms: Int)
$doc.endStatement();
$doc.startStatement(7, 0, 7, 109);
val conf = new SparkConf().setMaster("local[*]").setAppName("FramelessREPL").set("spark.ui.enabled", "false"); $doc.binder(conf, 7, 4, 7, 8)
$doc.endStatement();
$doc.startStatement(8, 0, 8, 95);
implicit val spark = SparkSession.builder().config(conf).appName("FramelessREPL").getOrCreate(); $doc.binder(spark, 8, 13, 8, 18)
$doc.endStatement();
$doc.startStatement(9, 0, 9, 38);
val res1 = spark.sparkContext.setLogLevel("WARN"); $doc.binder(res1, 9, 0, 9, 38)
$doc.endStatement();
$doc.startStatement(11, 0, 11, 42);
implicit val sqlContext = spark.sqlContext; $doc.binder(sqlContext, 11, 13, 11, 23)
$doc.endStatement();
$doc.startStatement(13, 0, 13, 24);
import spark.implicits._
$doc.endStatement();
$doc.startStatement(15, 0, 22, 1);
val apartments: Seq[Apartment] = Seq(
  Apartment("Paris", 50,  300000.0, 2),
  Apartment("Paris", 100, 450000.0, 3),
  Apartment("Paris", 25,  250000.0, 1),
  Apartment("Lyon",  83,  200000.0, 2),
  Apartment("Lyon",  45,  133000.0, 1),
  Apartment("Nice",  74,  325000.0, 3)
); $doc.binder(apartments, 15, 4, 15, 14)
$doc.endStatement();
$doc.endSection();

$doc.startSection();
$doc.startStatement(0, 0, 0, 60);
val aptTypedDs0 = TypedDataset.create[Apartment](apartments); $doc.binder(aptTypedDs0, 0, 4, 0, 15)
$doc.endStatement();
$doc.endSection();

$doc.startSection();
$doc.startStatement(0, 0, 0, 43);
val aptDs = spark.createDataset(apartments); $doc.binder(aptDs, 0, 4, 0, 9)
$doc.endStatement();
$doc.startStatement(1, 0, 1, 44);
val aptTypedDs1 = TypedDataset.create(aptDs); $doc.binder(aptTypedDs1, 1, 4, 1, 15)
$doc.endStatement();
$doc.endSection();
  }
}

error: /development/frameless/./docs/src/main/tut/FeatureOverviewTest.md: skipping file, the compiler produced no classfiles and reported no errors to explain what went wrong during compilation. Please report an issue to https://github.com/scalameta/mdoc/issues.

[olafurpg]

The repl.Session$ name is also referenced here

https://github.com/scalameta/mdoc/blob/e642727d5a32ca6f1126841d834b75cfb602507b/mdoc/src/main/scala/mdoc/internal/markdown/MarkdownCompiler.scala#L47

[vejeta]

repl.Session is something generated by mdoc, and not frameless. I am not acquainted enough with how mdoc works to cause it. Any hint?

[vejeta]

Issue still present with mdoc 2.1.0