asciidoctor / asciidoctor-doctest

:hammer: Test suite for Asciidoctor backends
MIT License
11 stars 16 forks source link

Test options don't work and test comments are stripped on `rake generate` (and rake test) #12

Closed obilodeau closed 7 years ago

obilodeau commented 7 years ago

To reproduce:

  1. create a simple test document:
// .stretch_class
// This example shows the impact of using the reveal.js special stretch CSS class
// :header_footer:
= Stretch class
  1. Try to rake generate the good starting output file:
bundle exec rake generate
  1. Look at the output file: comments and options are missing. The output is not honoring the :header_footer: option, neither on generation or tests.

I debugged up to regex parsing and it works fine up to there. Not sure what's going on.

jirutka commented 7 years ago

The problem is that currently only options from output examples are used. To be honest, I don’t remember why I did it in this way, it looks weird for me now. I’ll try to merge options from both input and output examples and see what happens.

Specifically header_footer is by default true for group names starting with document. So that’s probably the reason why you wrote that all examples must be in document.adoc?

jirutka commented 7 years ago

Fixed in dev.

obilodeau commented 7 years ago

Specifically header_footer is by default true for group names starting with document. So that’s probably the reason why you wrote that all examples must be in document.adoc?

No but close. When I first produced examples, they lacked the // .test-name required metadata so no output was generated. Then, since all of your examples are matching Asciidoctor's AST nodes I thought that my file names were the problem and that there was an assumption that all tests should match an AST node which I did with the metadata at the time. Only to figure out later that group names could be arbitrary.

Fixed in dev.

Good, I'll port my branch to use doctest dev and report back.

obilodeau commented 7 years ago

The example Rakefile in the README in the dev branch doesn't work. I'm getting this stack trace:

$ bundle exec rake doctest --trace
rake aborted!
NoMethodError: undefined method `output_suite=' for #<Asciidoctor::DocTest::RakeTasks:0x000000018ee300>
/home/olivier/src/asciidoc/asciidoctor-reveal.js/Rakefile:13:in `block in <top (required)>'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/bundler/gems/asciidoctor-doctest-13569fe28d36/lib/asciidoctor/doctest/rake_tasks.rb:100:in `initialize'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/Rakefile:12:in `new'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/Rakefile:12:in `<top (required)>'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/lib/rake/rake_module.rb:28:in `load'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/lib/rake/rake_module.rb:28:in `load_rakefile'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/lib/rake/application.rb:689:in `raw_load_rakefile'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/lib/rake/application.rb:94:in `block in load_rakefile'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/lib/rake/application.rb:93:in `load_rakefile'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/lib/rake/application.rb:77:in `block in run'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/lib/rake/application.rb:75:in `run'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/gems/rake-10.4.2/bin/rake:33:in `<top (required)>'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/bin/rake:22:in `load'
/home/olivier/src/asciidoc/asciidoctor-reveal.js/.bundle/gems/ruby/2.4.0/bin/rake:22:in `<top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:74:in `load'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:74:in `kernel_load'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/cli/exec.rb:27:in `run'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/cli.rb:360:in `exec'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor.rb:369:in `dispatch'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/cli.rb:20:in `dispatch'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/vendor/thor/lib/thor/base.rb:444:in `start'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/cli.rb:10:in `start'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/exe/bundle:35:in `block in <top (required)>'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
/usr/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/exe/bundle:27:in `<top (required)>'
/usr/bin/bundle:22:in `load'
/usr/bin/bundle:22:in `<main>'

Rakefile:

require 'asciidoctor/doctest'
require 'thread_safe'
require 'tilt'

DocTest::RakeTasks.new(:doctest) do |t|
  t.output_suite = DocTest::HTML::ExamplesSuite
  t.output_suite_opts = {
    examples_path: 'test/output/slim'
  }
  # add extra input examples (optional)
  t.input_suite_opts = {
    examples_path: [ *DocTest.examples_path, 'examples' ]
  }
  t.converter_opts = {
    template_dirs: 'templates/slim'
  }
end

I copied over and adapted the Rakefile content from features/fixtures/html-slim/Rakefile but I have new problems:

  1. it doesn't seem to honor the default tests in asciidoctor-doctest but only my own tests now.
  2. even after a doctest:generate FORCE=yes all my tests always fails, there is a difference in the output somehow. Here's the first example:
$ bundle exec rake doctest 

Running DocTest for the templates: templates/slim.

FFFFFFFasciidoctor: WARNING: <stdin>: line 30: section title out of sequence: expected level 2, got level 3
asciidoctor: WARNING: <stdin>: line 36: section title out of sequence: expected level 2, got level 3
FFFFFFFFFFFFFF

Aggregated results:

✗  Failure: concealed-slide-titles:concealed-slide-titles
   The three different ways to hide slide titles.

   E  <meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta charset="utf-8"><title>Concealed Slide Titles</title><meta content="yes" name="apple-mobile-web-app-capable"><meta content="black-translucent" name="apple-mobile-web-app-status-bar-style"><meta content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui" name="viewport"><link href="reveal.js/css/reveal.css" rel="stylesheet"><link href="reveal.js/css/theme/black.css" id="theme" rel="stylesheet"><link href="reveal.js/lib/css/zenburn.css" rel="stylesheet"><script>
   E    document.write( '<link rel="stylesheet" href="reveal.js/css/print/' + ( window.location.search.match( /print-pdf/gi ) ? 'pdf' : 'paper' ) + '.css" type="text/css" media="print">' );
   E  </script>
   E  <div class="reveal">
   E    <div class="slides">
   E      <section class="title" data-state="title">
   E        <h1>Concealed Slide Titles</h1>
   E      </section>
   E      <section>
   E        <div class="paragraph">
   E          <p>This</p>
   A  <!DOCTYPE html>
   A  <html lang="en">
   A    <head>
   A      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
   A      <meta charset="utf-8">
   A      <title>Concealed Slide Titles</title>
   A      <meta content="yes" name="apple-mobile-web-app-capable">
   A      <meta content="black-translucent" name="apple-mobile-web-app-status-bar-style">
   A      <meta content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui" name="viewport">
   A      <link href="reveal.js/css/reveal.css" rel="stylesheet">
   A      <link href="reveal.js/css/theme/black.css" id="theme" rel="stylesheet">
   A      <link href="reveal.js/lib/css/zenburn.css" rel="stylesheet">
   A      <script>
   A        document.write( '<link rel="stylesheet" href="reveal.js/css/print/' + ( window.location.search.match( /print-pdf/gi ) ? 'pdf' : 'paper' ) + '.css" type="text/css" media="print">' );
   A      </script>
   A    </head>
   A    <body>
   A      <div class="reveal">
   A        <div class="slides">
   A          <section class="title" data-state="title">
   A            <h1>Concealed Slide Titles</h1>
   A          </section>
   A          <section>
   A            <div class="paragraph">
   A              <p>This</p>
   A            </div>
   A          </section>
   A          <section id="_presentation">
   A            <div class="paragraph">
   A              <p>presentation’s titles</p>
   A            </div>
   A          </section>
   A          <section id="_concealed">
   A            <div class="paragraph">
   A              <p>should be concealed</p>
   A            </div>
   A          </section>
            </div>
   E      </section>
   E      <section id="_presentation">
   E        <div class="paragraph">
   E          <p>presentation’s titles</p>
   E        </div>
   E      </section>
   E      <section id="_concealed">
   E        <div class="paragraph">
   E          <p>should be concealed</p>
   E        </div>
   E      </section>
   E    </div>
   E  </div>
   E  <script src="reveal.js/lib/js/head.min.js"></script><script src="reveal.js/js/reveal.js"></script><script>
   E    // See https://github.com/hakimel/reveal.js#configuration for a full list of configuration options
   E    Reveal.initialize({
   E      // Display controls in the bottom right corner
   E      controls: true,
   E      // Display a presentation progress bar
   E      progress: true,
   E      // Display the page number of the current slide
   E      slideNumber: false,
   E      // Push each slide change to the browser history
   E      history: false,
   E      // Enable keyboard shortcuts for navigation
   E      keyboard: true,
   E      // Enable the slide overview mode
   E      overview: true,
   E      // Vertical centering of slides
   E      center: true,
   E      // Enables touch navigation on devices with touch input
   E      touch: true,
   E      // Loop the presentation
   E      loop: false,
   E      // Change the presentation direction to be RTL
   E      rtl: false,
   E      // Turns fragments on and off globally
   E      fragments: true,
   E      // Flags if the presentation is running in an embedded mode,
   E      // i.e. contained within a limited portion of the screen
   E      embedded: false,
   E      // Number of milliseconds between automatically proceeding to the
   E      // next slide, disabled when set to 0, this value can be overwritten
   E      // by using a data-autoslide attribute on your slides
   E      autoSlide: 0,
   E      // Stop auto-sliding after user input
   E      autoSlideStoppable: true,
   E      // Enable slide navigation via mouse wheel
   E      mouseWheel: false,
   E      // Hides the address bar on mobile devices
   E      hideAddressBar: true,
   E      // Opens links in an iframe preview overlay
   E      previewLinks: false,
   E      // Theme (e.g., beige, black, league, night, serif, simple, sky, solarized, white)
   E      // NOTE setting the theme in the config no longer works in reveal.js 3.x
   E      //theme: Reveal.getQueryHash().theme || 'black',
   E      // Transition style (e.g., none, fade, slide, convex, concave, zoom)
   E      transition: Reveal.getQueryHash().transition || 'slide',
   E      // Transition speed (e.g., default, fast, slow)
   E      transitionSpeed: 'default',
   E      // Transition style for full page slide backgrounds (e.g., none, fade, slide, convex, concave, zoom)
   E      backgroundTransition: 'fade',
   E      // Number of slides away from the current that are visible
   E      viewDistance: 3,
   E      // Parallax background image (e.g., "'https://s3.amazonaws.com/hakim-static/reveal-js/reveal-parallax-1.jpg'")
   E      parallaxBackgroundImage: '',
   E      // Parallax background size in CSS syntax (e.g., "2100px 900px")
   E      parallaxBackgroundSize: '',
   E    
   E      // The "normal" size of the presentation, aspect ratio will be preserved
   E      // when the presentation is scaled to fit different resolutions. Can be
   E      // specified using percentage units.
   E      width: 960,
   E      height: 700,
   E    
   E      // Factor of the display size that should remain empty around the content
   E      margin: 0.1,
   E    
   E      // Bounds for smallest/largest possible scale to apply to content
   E      minScale: 0.2,
   E      maxScale: 1.5,
   E    
   E      // Optional libraries used to extend on reveal.js
   E      dependencies: [
   E          { src: 'reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } },
   E          { src: 'reveal.js/plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
   E          { src: 'reveal.js/plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
   E    
   E          { src: 'reveal.js/plugin/zoom-js/zoom.js', async: true, condition: function() { return !!document.body.classList; } },
   E          { src: 'reveal.js/plugin/notes/notes.js', async: true, condition: function() { return !!document.body.classList; } }
   E      ]
   E    });
   E  </script>
   A      </div>
   A      <script src="reveal.js/lib/js/head.min.js"></script><script src="reveal.js/js/reveal.js"></script><script>
   A        // See https://github.com/hakimel/reveal.js#configuration for a full list of configuration options
   A        Reveal.initialize({
   A          // Display controls in the bottom right corner
   A          controls: true,
   A          // Display a presentation progress bar
   A          progress: true,
   A          // Display the page number of the current slide
   A          slideNumber: false,
   A          // Push each slide change to the browser history
   A          history: false,
   A          // Enable keyboard shortcuts for navigation
   A          keyboard: true,
   A          // Enable the slide overview mode
   A          overview: true,
   A          // Vertical centering of slides
   A          center: true,
   A          // Enables touch navigation on devices with touch input
   A          touch: true,
   A          // Loop the presentation
   A          loop: false,
   A          // Change the presentation direction to be RTL
   A          rtl: false,
   A          // Turns fragments on and off globally
   A          fragments: true,
   A          // Flags if the presentation is running in an embedded mode,
   A          // i.e. contained within a limited portion of the screen
   A          embedded: false,
   A          // Number of milliseconds between automatically proceeding to the
   A          // next slide, disabled when set to 0, this value can be overwritten
   A          // by using a data-autoslide attribute on your slides
   A          autoSlide: 0,
   A          // Stop auto-sliding after user input
   A          autoSlideStoppable: true,
   A          // Enable slide navigation via mouse wheel
   A          mouseWheel: false,
   A          // Hides the address bar on mobile devices
   A          hideAddressBar: true,
   A          // Opens links in an iframe preview overlay
   A          previewLinks: false,
   A          // Theme (e.g., beige, black, league, night, serif, simple, sky, solarized, white)
   A          // NOTE setting the theme in the config no longer works in reveal.js 3.x
   A          //theme: Reveal.getQueryHash().theme || 'black',
   A          // Transition style (e.g., none, fade, slide, convex, concave, zoom)
   A          transition: Reveal.getQueryHash().transition || 'slide',
   A          // Transition speed (e.g., default, fast, slow)
   A          transitionSpeed: 'default',
   A          // Transition style for full page slide backgrounds (e.g., none, fade, slide, convex, concave, zoom)
   A          backgroundTransition: 'fade',
   A          // Number of slides away from the current that are visible
   A          viewDistance: 3,
   A          // Parallax background image (e.g., "'https://s3.amazonaws.com/hakim-static/reveal-js/reveal-parallax-1.jpg'")
   A          parallaxBackgroundImage: '',
   A          // Parallax background size in CSS syntax (e.g., "2100px 900px")
   A          parallaxBackgroundSize: '',
   A        
   A          // The "normal" size of the presentation, aspect ratio will be preserved
   A          // when the presentation is scaled to fit different resolutions. Can be
   A          // specified using percentage units.
   A          width: 960,
   A          height: 700,
   A        
   A          // Factor of the display size that should remain empty around the content
   A          margin: 0.1,
   A        
   A          // Bounds for smallest/largest possible scale to apply to content
   A          minScale: 0.2,
   A          maxScale: 1.5,
   A        
   A          // Optional libraries used to extend on reveal.js
   A          dependencies: [
   A              { src: 'reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } },
   A              { src: 'reveal.js/plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
   A              { src: 'reveal.js/plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
   A        
   A              { src: 'reveal.js/plugin/zoom-js/zoom.js', async: true, condition: function() { return !!document.body.classList; } },
   A              { src: 'reveal.js/plugin/notes/notes.js', async: true, condition: function() { return !!document.body.classList; } }
   A          ]
   A        });
   A      </script>
   A    </body>
   A  </html>

Any pointers, fixes or ideas would be appreciated.

obilodeau commented 7 years ago

If you want to test it out, I just pushed the current adaptation in this PR: https://github.com/asciidoctor/asciidoctor-reveal.js/pull/116

Checkout this code and with bundler install you should be good to go. However, I prefer to use these:

bundle config --local github.https true
bundle --path=.bundle/gems --binstubs=.bundle/.bin
jirutka commented 7 years ago

The example Rakefile in the README in the dev branch doesn't work.

Sorry, I forgot to update README when changed RakeTasks. I’ve updated it now (ef272bffc529731bf707a92471ebdae1a414b783).

You can also look into asciidoctor-html5s for working example.

it doesn't seem to honor the default tests in asciidoctor-doctest but only my own tests now.

When you redefine path in input_examples, then you must explicitly add DocTest.examples_path into it if you want to include reference examples.

  t.input_examples :asciidoc, path: [
    *DocTest.examples_path,
    'test/examples/asciidoc'
  ]

even after a doctest:generate FORCE=yes all my tests always fails, there is a difference in the output somehow.

I think that it’s caused by doctype declaration in the output, but don’t know where exactly is the problem. I’ll look at it later.

However, asserting complete HTML including boilerplate in all examples is inconvenient. You should select only relevant part for each example – with XPath using options include and exclude (→Readme, →example). BTW that’s why header_footer is disabled by default for all but document*:* and why inline_*:* sets :include: ./p/node() by default (can be overriden by paragraph_xpath, see Readme).

obilodeau commented 7 years ago

Re-added default with your example. Works like a charm. :+1:

I think that it’s caused by doctype declaration in the output, but don’t know where exactly is the problem. I’ll look at it later.

I'll file a separate issue.

You should select only relevant part for each example – with XPath using options include and exclude

Oh that's really neat! Most of my tests properly scoped now. Default document tests are still failing though.

Original issue is solved to my satisfaction. If you could tag a release and publish it on rubygem that would be great!