scaladoc tool : argument files (@-files)

michelou commented 3 years ago

In Scala 2 command line argument files (@-files) are supported by scalac and scaladoc.
In Scala 3 only scalac supports @-files (although undocumented, see foreword below).

Foreword - Scala 2 versus Scala 3 help message

Scala 2 output (note the last output line) :

user@host MINGW64 /w/dotty
$ /c/opt/scala-2.13.4/bin/scalac -help 2>&1| grep file
Usage: scalac <options> <source files>
  -bootclasspath <path>        Override location of bootstrap class files.
  -classpath <path>            Specify where to find user class files.
  -d <directory|jar>           destination for generated classfiles.
  -dependencyfile <file>       Set dependency tracking file.
  -enchostg <enchostg>         Specify character enchostg used by source files.
  -opt-inline-from:<patterns>  Patterns for classfile names from which to allow inlining, `help` for details.
  -sourcepath <path>           Specify location(s) of source files.
  -target:<target>             Target platform for object files. ([8],9,10,11,12)
  @<file>                      A text file containing compiler arguments (options and source files) [false]

Scala 3 output :

user@host MINGW64 /w/dotty
$ /c/opt/scala-3.0.0-RC1/bin/scalac -help 2>&1| grep file
Usage: scalac <options> <source files>
-bootclasspath                                    Override location of bootstrap class files.
-classpath                                        Specify where to find user class files.
-d                                                Destination for generated classfiles.
-enchostg                                         Specify character enchostg used by source files.
-from-tasty                                       Compile classes from tasty files. The arguments are .tasty or .jar files.
-project-logo                                     The file that contains the project's logo (in /images).
-semanticdb-target                                Specify an alternative output directory for SemanticDB files.
-siteroot                                         A directory containing static files from which to generate documentation.
-sourcepath                                       Specify location(s) of source files.

`scalac` usage example - Scala 3

user@host MINGW64 /w/dotty
$ export JAVA_HOME=/c/opt/jdk-1.8.0_282-b08/

user@host MINGW64 /w/dotty
$ dir c:\\temp\\classes

user@host MINGW64 /w/dotty
$ cat scalac_opts.txt
-d c:\\temp\\classes tests\\pos\\HelloWorld.scala

user@host MINGW64 /w/dotty
$ /c/opt/scala-3.0.0-RC1/bin/scalac @scalac_opts.txt

user@host MINGW64 /w/dotty
$ dir c:\\temp\\classes
HelloWorld$.class  HelloWorld.class  HelloWorld.tasty

user@host MINGW64 /w/dotty
$ /c/opt/scala-3.0.0-RC1/bin/scala -cp c:\\temp\\classes HelloWorld
hello world

`scaladoc` usage example - Scala 2 versus Scala 3

user@host MINGW64 /w/dotty
$ rm -rf c:\\temp\\docs\*

user@host MINGW64 /w/dotty
$ cat scaladoc_opts.txt
-d c:\\temp\\docs tests\\pos\\HelloWorld.scala

user@host MINGW64 /w/dotty
$ /c/opt/scala-2.13.4/bin/scaladoc @scaladoc_opts.txt

user@host MINGW64 /w/dotty
$ dir c:\\temp\\docs
HelloWorld$.html  index.html  index.js  lib

user@host MINGW64 /w/dotty
$ rm -rf c:\\temp\\docs\*

user@host MINGW64 /w/dotty
$ /c/opt/scala-3.0.0-RC1/bin/scaladoc @scaladoc_opts.txt
scaladoc will ignore following non-existent paths: @scaladoc_opts.txt
Destination is not provided, please provide '-d' parameter pointing to directory where docs should be created
Failure

Expectation

Add support for @-files to scaladoc.

michelou commented 3 years ago

@BarkingBad As of February 20 Scala 3 output looks as follows with PR #11476 :

user@host MINGW64 /w/dotty
$ cat scaladoc_opts.txt
-d c:\\temp\\docs tests\\pos\\HelloWorld.scala

user@host MINGW64 /w/dotty
$ /c/opt/scala-3.0.0-RC2-bin-SNAPSHOT/bin/scaladoc @scaladoc_opts.txt
scaladoc suports only .tasty and .jar files, following files will be ignored: tests\pos\HelloWorld.scala

user@host MINGW64 /w/dotty
$ dir c:\\temp\\docs
api  favicon.ico  fonts  hljs  images  scaladoc.version  scripts  styles

Note: In the above the output directory docs\ contains an empty documentation.

2nd attempt with a .tasty file :

user@odin MINGW64 /w/dotty
$ cat scaladoc_opts1.txt
-d c:\\temp\\docs c:\\temp\\classes\\HelloWorld.tasty

user@host MINGW64 /w/dotty
$ /c/opt/scala-3.0.0-RC2-bin-SNAPSHOT/bin/scaladoc @scaladoc_opts1.txt
java.nio.file.InvalidPathException: Illegal char <<> at index 4: api/<empty>/HelloWorld$.html
        at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
        at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
        at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
        at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
        at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
        at java.base/java.nio.file.Path.resolve(Path.java:515)
        at dotty.tools.scaladoc.renderers.Writer.dest(Writer.scala:17)
        at dotty.tools.scaladoc.renderers.Writer.write(Writer.scala:23)
        at dotty.tools.scaladoc.renderers.Writer.write$(Writer.scala:13)
        at dotty.tools.scaladoc.renderers.HtmlRenderer.write(HtmlRenderer.scala:26)
        at dotty.tools.scaladoc.renderers.HtmlRenderer.renderPage(HtmlRenderer.scala:85)
        at dotty.tools.scaladoc.renderers.HtmlRenderer.renderPage$$anonfun$1(HtmlRenderer.scala:85)
        at scala.collection.immutable.List.flatMap(List.scala:293)
        at scala.collection.immutable.List.flatMap(List.scala:79)
        at dotty.tools.scaladoc.renderers.HtmlRenderer.renderPage(HtmlRenderer.scala:85)
        at dotty.tools.scaladoc.renderers.HtmlRenderer.renderPage$$anonfun$1(HtmlRenderer.scala:85)
        at scala.collection.immutable.List.flatMap(List.scala:293)
        at scala.collection.immutable.List.flatMap(List.scala:79)
        at dotty.tools.scaladoc.renderers.HtmlRenderer.renderPage(HtmlRenderer.scala:85)
        at dotty.tools.scaladoc.renderers.HtmlRenderer.$anonfun$9(HtmlRenderer.scala:114)
        at scala.collection.immutable.List.map(List.scala:246)
        at scala.collection.immutable.List.map(List.scala:79)
        at dotty.tools.scaladoc.renderers.HtmlRenderer.render(HtmlRenderer.scala:114)
        at dotty.tools.scaladoc.Scaladoc$.run(Scaladoc.scala:177)
        at dotty.tools.scaladoc.Scaladoc$.run(Scaladoc.scala:65)
        at dotty.tools.scaladoc.Main.run(Main.scala:18)
        at dotty.tools.scaladoc.Main$.main(Main.scala:24)
        at dotty.tools.scaladoc.Main.main(Main.scala)

romanowski commented 3 years ago

@michelou scaladoc does not support .scala files at this moment, it only works with .tasty, .jar files or directories (that should contain .tasty files). Scaladoc (in Scala 3.x) is based on tasty and does not trigger compiler (we may change that but in most cases, docs are called after code is compiled).

I can definitely see that we have a problem with entries from the default top-level package (no package defined in .scala file) on Windows. We will fix that.

michelou commented 3 years ago

@romanowski Right. That's the reason for my 2nd attempt with @-filescaladoc_opts1.txt :innocent:

michelou commented 3 years ago

@romanowski In my case the following quick fix solves the above java.nio.file.InvalidPathException error :

$ git diff scaladoc\src\dotty\tools\scaladoc\renderers\Writer.scala
diff --git a/scaladoc/src/dotty/tools/scaladoc/renderers/Writer.scala b/scaladoc/src/dotty/tools/scaladoc/renderers/Writer.scala
index dad7aaa9e6..1a60921f3a 100644
--- a/scaladoc/src/dotty/tools/scaladoc/renderers/Writer.scala
+++ b/scaladoc/src/dotty/tools/scaladoc/renderers/Writer.scala
@@ -6,6 +6,7 @@ import java.nio.file.Paths
 import java.nio.file.Path
 import java.nio.file.Files
 import java.io.File
+import java.util.regex.Pattern

 import util.HTML._

@@ -13,8 +14,13 @@ import util.HTML._
 trait Writer(using ctx: DocContext) extends Locations:
   private val args = summon[DocContext].args

+  // We create one pattern since we are applying the same regex to many strings.
+  private val emptyPattern = Pattern.compile("<empty>")
+
   private def dest(path: String) =
-    val absPath = args.output.toPath.resolve(path)
+    val m = emptyPattern.matcher(path)
+    val path1 = if m.find() then m.replaceAll("_empty_") else path
+    val absPath = args.output.toPath.resolve(path1)
     if !Files.exists(absPath.getParent) then Files.createDirectories(absPath.getParent)
     absPath

NB. In the above I choose to replace occurences of <empty> in destination path with _empty_.

Now scaladoc produces the expected output :

user@host W:\dotty
$ ls c:\temp\docs

user@host W:\dotty
$ type scaladoc_opts1.txt
-d c:\\temp\\docs c:\\temp\\classes\\HelloWorld.tasty

user@host W:\dotty
$ c:\opt\scala-3.0.0-RC2-bin-SNAPSHOT\bin\scaladoc @scaladoc_opts1.txt

user@host W:\dotty
$ ls c:\temp\docs c:\temp\docs\api c:\temp\docs\api\_empty_
'c:\temp\docs':
api  favicon.ico  fonts  hljs  images  scaladoc.version  scripts  styles

'c:\temp\docs\api':
_empty_  index.html

'c:\temp\docs\api\_empty_':
'HelloWorld$.html'

mkurz commented 1 year ago

scaladoc does not support .scala files at this moment, it only works with .tasty, .jar files or directories (that should contain .tasty files). Scaladoc (in Scala 3.x) is based on tasty and does not trigger compiler (we may change that but in most cases, docs are called after code is compiled).

Will .scala files be supported by scaladoc anytime soon? Also see #18215

scala / scala3