Does asciidoctorj v1.5.7 support asciidoctorj-pdf v1.5.0-alpha.16?

[charleech]

I'm testing the asciidoctorj v1.5.7 with asciidoctor-pdf-example by changing the pom.xmlas the following: -

First try

<asciidoctorj.version>1.5.7</asciidoctorj.version>
<jruby.version>9.1.16.0</jruby.version>

There are some error display at console

uri:classloader:/gems/prawn-svg-0.27.1/lib/prawn/svg/color.rb:212: warning: shadowing outer local variable - result
uri:classloader:/gems/prawn-svg-0.27.1/lib/prawn/svg/properties.rb:50: warning: `*' interpreted as argument prefix
uri:classloader:/gems/ttfunk-1.5.1/lib/ttfunk/table.rb:26: warning: instance variable @offset not initialized
...
uri:classloader:/gems/prawn-2.2.2/lib/prawn/font/ttf.rb:118: warning: instance variable @italic_angle not initialized
uri:classloader:/gems/ttfunk-1.5.1/lib/ttfunk/table.rb:26: warning: instance variable @offset not initialized
...
uri:classloader:/gems/prawn-2.2.2/lib/prawn/font/ttf.rb:118: warning: instance variable @italic_angle not initialized
uri:classloader:/gems/ttfunk-1.5.1/lib/ttfunk/table.rb:26: warning: instance variable @offset not initialized
...

Second try

<asciidoctorj.version>1.5.7</asciidoctorj.version>
<jruby.version>9.1.16.0</jruby.version>

<sourceHighlighter>rouge</sourceHighlighter>

There are some error display at console

uri:classloader:/gems/prawn-svg-0.27.1/lib/prawn/svg/color.rb:212: warning: shadowing outer local variable - result
uri:classloader:/gems/prawn-svg-0.27.1/lib/prawn/svg/properties.rb:50: warning: `*' interpreted as argument prefix
uri:classloader:/gems/rouge-2.0.7/lib/rouge/lexers/bsl.rb:62: warning: Ambiguous first argument; make sure.
...
uri:classloader:/gems/rouge-2.0.7/lib/rouge/lexers/io.rb:28: warning: Ambiguous first argument; make sure.
...
uri:classloader:/gems/rouge-2.0.7/lib/rouge/lexers/perl.rb:68: warning: Ambiguous first argument; make sure
...
uri:classloader:/gems/rouge-2.0.7/lib/rouge/lexers/matlab.rb:30: warning: Ambiguous first argument; make sure.
...
uri:classloader:/gems/rouge-2.0.7/lib/rouge/lexers/jinja.rb:39: warning: Ambiguous first argument; make sure.
...
uri:classloader:/gems/rouge-2.0.7/lib/rouge/lexers/qml.rb:19: warning: Ambiguous first argument; make sure.
...
uri:classloader:/gems/rouge-2.0.7/lib/rouge/lexers/javascript.rb:34: warning: Ambiguous first argument; make sure.
...
uri:classloader:/gems/rouge-2.0.7/lib/rouge/lexers/scala.rb:38: warning: Ambiguous first argument; make sure.
... too many languages ....
uri:classloader:/gems/ttfunk-1.5.1/lib/ttfunk/table.rb:26: warning: instance variable @offset not initialized
...
uri:classloader:/gems/prawn-2.2.2/lib/prawn/font/ttf.rb:118: warning: instance variable @italic_angle not initialized
uri:classloader:/gems/ttfunk-1.5.1/lib/ttfunk/table.rb:26: warning: instance variable @offset not initialized
...
uri:classloader:/gems/prawn-2.2.2/lib/prawn/font/ttf.rb:118: warning: instance variable @italic_angle not initialized
uri:classloader:/gems/ttfunk-1.5.1/lib/ttfunk/table.rb:26: warning: instance variable @offset not initialized
...

Does asciidoctorj v1.5.7 support asciidoctorj-pdf v1.5.0-alpha.16?

[robertpanzer]

These messages are warnings from JRuby complaining about the Ruby code that is in the used libraries, but that shouldn't result in the document not being created. Do you get a PDF, or does the Maven build fail with some error?

[charleech]

Except the long printing logs, the maven build successfully and I can get the proper PDF.

Every source code inside it are highlighted.

[charleech]

Furthermore, when I use include with tag as the following: -

[source, java, linenums]
.Some example code
----
include::Hello.java[tag=hello]
----

The Hello.java

// tag::hello[]
public class Test {
  public static void main(String[] args) {
    System.out.println("Hello World!"); 
  }
}
// end::hello[]

There is a warning as

WARNING: some-file: line 34: detected unclosed tag 'hello' starting at line 2 of include file:Hello.java

The PDF highlighted output are including that end tag as

public class Test {
  public static void main(String[] args) {
    System.out.println("Hello World!"); 
  }
}
// end::hello[] <-- This should not be display in PDF

[mojavelinux]

Something tells me it's because the tag is on the last line of the file. It shouldn't be a problem, but for some reason it is. Then again, you don't really need a tag in this case since you're including the whole file.

Can you run oc -d on that file (or whatever command will output the contents in binary?)

[charleech]

The od -c is

0000000   /   /       t   a   g   :   :   h   e   l   l   o   [   ]  \r
0000020  \n   p   u   b   l   i   c       c   l   a   s   s       T   e
0000040   s   t       {  \r  \n           p   u   b   l   i   c       s
0000060   t   a   t   i   c       v   o   i   d       m   a   i   n   (
0000100   f   i   n   a   l       S   t   r   i   n   g   [   ]       a
0000120   r   g   s   )       {  \r  \n                   S   y   s   t
0000140   e   m   .   o   u   t   .   p   r   i   n   t   l   n   (   "
0000160   H   e   l   l   o       W   o   r   l   d   !   "   )   ;  \r
0000200  \n           }  \r  \n   }  \r  \n   /   /       e   n   d   :
0000220   :   h   e   l   l   o   [   ]
0000230

The od -d is

0000000 12079 29728 26465 14906 25960 27756 23407  3421
0000020 28682 25205 26988  8291 27747 29537  8307 25940
0000040 29811 31520  2573  8224 30064 27746 25449 29472
0000060 24948 26996  8291 28534 25705 27936 26977 10350
0000100 26982 24942  8300 29779 26994 26478 23899 24864
0000120 26482 10611 31520  2573  8224  8224 31059 29811
0000140 28005 28462 29813 28718 26994 29806 28268  8744
0000160 25928 27756  8303 28503 27762  8548 10530  3387
0000200  8202 32032  2573  3453 12042  8239 28261 14948
0000220 26682 27749 28524 23899
0000230

The od -xc is

0000000    2f2f    7420    6761    3a3a    6568    6c6c    5b6f    0d5d
          /   /       t   a   g   :   :   h   e   l   l   o   [   ]  \r
0000020    700a    6275    696c    2063    6c63    7361    2073    6554
         \n   p   u   b   l   i   c       c   l   a   s   s       T   e
0000040    7473    7b20    0a0d    2020    7570    6c62    6369    7320
          s   t       {  \r  \n           p   u   b   l   i   c       s
0000060    6174    6974    2063    6f76    6469    6d20    6961    286e
          t   a   t   i   c       v   o   i   d       m   a   i   n   (
0000100    6966    616e    206c    7453    6972    676e    5d5b    6120
          f   i   n   a   l       S   t   r   i   n   g   [   ]       a
0000120    6772    2973    7b20    0a0d    2020    2020    7953    7473
          r   g   s   )       {  \r  \n                   S   y   s   t
0000140    6d65    6f2e    7475    702e    6972    746e    6e6c    2228
          e   m   .   o   u   t   .   p   r   i   n   t   l   n   (   "
0000160    6548    6c6c    206f    6f57    6c72    2164    2922    0d3b
          H   e   l   l   o       W   o   r   l   d   !   "   )   ;  \r
0000200    200a    7d20    0a0d    0d7d    2f0a    202f    6e65    3a64
         \n           }  \r  \n   }  \r  \n   /   /       e   n   d   :
0000220    683a    6c65    6f6c    5d5b
          :   h   e   l   l   o   [   ]

[mojavelinux]

Yep, this is a bug in Asciidoctor when the tag occurs on the last line of the file and the file does have a trailing newline. It seems we're missing a test case for this scenario. I'll pursue this upstream. In the meantime, you can solve the problem by adding a trailing newline to the Java file.

[mojavelinux]

See https://github.com/asciidoctor/asciidoctor/issues/2830

Thanks for reporting @charleech!

[charleech]

Thank you. I can confirm that adding new line at the end of file solve this.

Anyhow, how about the JRuby warning? How can I get rid of if?

[mojavelinux]

It looks like AsciidoctorJ enables verbose mode by default, which causes Ruby to report these warnings.

https://github.com/asciidoctor/asciidoctorj/blob/f8c08a782230a37e54f04174389bbf326b34fa76/asciidoctorj-core/src/main/resources/org/asciidoctor/internal/asciidoctorclass.rb

I don't think that should be hardcoded. That should be an option that is set when the Asciidoctor instance is configured. Although the warnings are harmless, it's not always desirable to have this setting on. First because of pedandic Ruby warnings. But also because some validations in Asciidoctor are still false positives and you may not want them on in a production system / pipeline.

I think we'll need a more specific issue to address this change request.

[robertpanzer]

Oops, I wasn’t aware that this enables these warnings. I added setting $VERBOSE to true because this seemed to be the only way to get warnings about invalid refs, which imo a user always wants to see.

[mojavelinux]

It's true that it does need to be set for that outcome, but it should be managed. I recommend logic similar to that used by the CLI's invoker: https://github.com/asciidoctor/asciidoctor/blob/master/lib/asciidoctor/cli/invoker.rb#L65-L76

The user doesn't always want to see the invalid refs warning because it's still not always accurate. It's certainly much better, but there are still times when it's overzealous (due to how inline references are handled).

[robertpanzer]

I had a look at that while implementing the logger. But it was a bit confusing to me: We implemented loggers with log levels that allow to control the amount of information you get, but need to set a global variable to get such warnings. Would it make sense to log with a different progname then?

Or would it make sense to control this kind of logging with another variable? Controlling interpreter warnings from JRuby and invalid ref warnings from Asciidoctor with the same global variable comes unexpected.

[robertpanzer]

Also the verbose mode of AsciidoctorJ historically seems to work a bit different than the Asciidoctor CLI. It also seems to print out timings. Should I remove this and align with the Asciidoctor CLI?

[mojavelinux]

Also the verbose mode of AsciidoctorJ historically seems to work a bit different than the Asciidoctor CLI. It also seems to print out timings.

The Asciidoctor CLI used to do this, but then moved the timings to the -t, --timings flag. So yes, I recommend aligning with it.

[robertpanzer]

The Asciidoctor CLI used to do this, but then moved the timings to the -t, --timings flag. So yes, I recommend aligning with it.

Ah, good to hear this, I was already wondering about the differences in behavior between asciidoctor and asciidoctorj. I'll create PRs to align it.

[mojavelinux]

Since this issue of verbose mode has also come up in the Gradle plugin, I think we need a dedicated issue to rethink how to handle it properly. Specifically, there's confusion about debug mode in the logger and Ruby's verbose mode, which has been complicated by the fact that I associated one of the warnings in Asciidoctor core with verbose mode instead of the debug level. But there's still value in being able to control verbose mode, even from the API. But we should view it more as an environment setting (like the gem path). @robertpanzer could you open a new issue to discuss how to control the value of the $VERBOSE global variable through the Asciidoctor API (perhaps at initialization time)?