Built-in html formatter generates a blank html report

[mangjoze]

Describe the bug When running my test using built-in html formatter, it generates a blank HTML file. Viewing the source of the html, I see a bunch of css and license stuff.

To Reproduce Steps to reproduce the behavior:

1. Execute cucumber --format HTML --out report.html
2. Open the html report and html file is blank.

Expected behavior HTML report to be generated should not be blank.

Context & Motivation I can't report my test runs.

Environment cucumber 6.0.0. ruby 2.7.0 Windows 10

[aurelien-reeves]

I tried to reproduce without success.

I have been able to generate a working HTML report.

Would you have a minimal reproduction scenario? What is your browser? Do you have any kind of add-on which may block execution of javascript?

[mattwynne]

@mangjoze you should know that the html formatter doesn't generate a static HTML document. Rather, it generates an HTML page containing JSON data about your test run, and some JavaScript that should be able to render that JSON data into a nice-looking report. If the page is blank then it's probably because the JavaScript is failing for some reason.

Can you check the developer tools console to see if there are any errors?

[mangjoze]

@aurelien-reeves @mattwynne I should have mentioned in the ticket that I'm pretty new at cucumber so I might be doing things the wrong way. I have tried googling for some kind of solution but to no avail hence the ticket here.

Would you have a minimal reproduction scenario? I'll try to make a minimal feature/step file and see if I could repro it. I will try and provide the html file here (need to confirm if I could put it here due to confidentiality reasons)

What is your browser? Browser is Chrome/IE/Edge

Do you have any kind of add-on which may block execution of javascript? Can you check the developer tools console to see if there are any errors? Not that I am aware of but I will check as soon as I get back to the office.

Thank you guys for the help! Much appreciated.

[mattwynne]

A minimal example to reproduce would be perfect, thanks @mangjoze!

[mattwynne]

Also...

I'm pretty new at cucumber

...welcome to the community! 😄 👋 🎉

[mangjoze]

I did a minimal feature/step file using accuweather and it worked. So I'm not quite sure why it does not work on my work feature/step files. Also, on the accuweather feature file, the embed doesn't work for me. I get an undefined method 'embed' (NoMethodError)'.

[aslakhellesoy]

The embed method has been renamed to attach: https://github.com/cucumber/cucumber-ruby/blob/master/CHANGELOG.md#removed-1

I assume this will make it work for you. If not, please make an mcve and we'll reopen this issue.

[mattwynne]

@aslakhellesoy I'm trying to understand why the OP got a blank HTML report due to this error of using the wrong method to add an attachment. Can you explain what you think was happening?

[aurelien-reeves]

I am not sure the blank report and the use of the embed method are related: it seems that with accuweather there were a "undefined" 'embed' method, but the report was not blank.

@mangjoze still, we would need a minimal reproducible scenario. Try to isolate the step responsible for your issue

Did you take a look in your browser console with a blank report if there are JavaScript errors?

[mangjoze]

@aurelien-reeves

@mangjoze still, we would need a minimal reproducible scenario. Try to isolate the step responsible for your issue I'm still working on it, so I managed to make it work on my test feature files by creating a new project and only included the feature files with completed step definitions. What I noticed when I first logged this is that, I am only running the @wip profile but it seems to include in the html source, the other feature files I have written which has either incomplete or no step definitions. Is that something that would prevent the html report to load? When there are feature files with no step def/incomplete ones?

Did you take a look in your browser console with a blank report if there are JavaScript errors?

No Javascript errors

[aurelien-reeves]

I'm still working on it, so I managed to make it work on my test feature files by creating a new project and only included the feature files with completed step definitions. What I noticed when I first logged this is that, I am only running the @wip profile but it seems to include in the html source, the other feature files I have written which has either incomplete or no step definitions. Is that something that would prevent the html report to load? When there are feature files with no step def/incomplete ones?

That's a trail :)

No Javascript errors

Ok, thanks

[marnen]

@mattwynne:

Rather, it generates an HTML page containing JSON data about your test run, and some JavaScript that should be able to render that JSON data into a nice-looking report.

Unfortunately, the way that’s done makes the Cucumber HTML report much less useful than it used to be. Back in the day, Cucumber used to use a similar approach to RSpec: it wrote straight HTML as scenarios ran, so that the file would render something useful in the browser before it was completely written. That makes it ideal for monitoring a long run with lots of scenarios.

Current Cucumber HTML output, though, renders nothing until the run is complete (I think because window.CUCUMBER_MESSAGES isn’t well-formed JSON/JS till the final closing bracket is written), which means I need to wait for the entire run to finish before I see any output. Is there an HTML formatter for current Cucumber that doesn’t have this drawback? I haven’t found one, and I really miss the old RSpec-like behavior. (And if this is off-topic, I’ll raise a new issue or ask on the list.)

[mattwynne]

it wrote straight HTML as scenarios ran, so that the file would render something useful in the browser before it was completely written. That makes it ideal for monitoring a long run with lots of scenarios.

In fact, this kind of live monitoring of a test run from a browser should now be easier to do, given that the results are streamed out as little ndjson packets. The problem would be getting that stream from your Cucumber process to your browser. It's a really cool idea, but yes it is off-topic for this thread I think :) If you're interested in pursuing it, let's create a new issue (in https://github.com/cucumber/cucumber) and discuss it there.

[pgundlupetvenkatesh]

I was able to reproduce this with Cucumber --tags. bundle exec cucumber -t '@<tag_name>' -f html -o report.html would throw a blank report but when I ran it with a Cucumber file name bundle exec cucumber features/<file_name>.feature -f html -o report.html I was able to view the HTML report and was not blank.

Did a quick check in the DOM Console and found this

My bundle list

Gems included by the bundle:
  * actionview (6.1.7)
  * activesupport (6.1.7)
  * addressable (2.8.1)
  * akami (1.3.1)
  * awesome_print (1.9.2)
  * binding_of_caller (1.0.0)
  * builder (3.2.4)
  * byebug (11.1.3)
  * certified (1.0.0)
  * childprocess (4.1.0)
  * clipboard (1.3.6)
  * coderay (1.1.3)
  * concurrent-ruby (1.1.10)
  * crass (1.0.6)
  * cucumber (6.1.0)
  * cucumber-core (9.0.1)
  * cucumber-create-meta (4.0.0)
  * cucumber-cucumber-expressions (12.1.3)
  * cucumber-gherkin (18.1.1)
  * cucumber-html-formatter (13.0.0)
  * cucumber-messages (15.0.0)
  * cucumber-tag-expressions (3.0.1)
  * cucumber-wire (5.0.1)
  * data_magic (1.2)
  * date (3.3.3)
  * debug_inspector (1.1.0)
  * diff-lcs (1.5.0)
  * erubi (1.12.0)
  * faker (2.22.0)
  * ffi (1.15.5)
  * gyoku (1.4.0)
  * httpi (3.0.1)
  * i18n (1.12.0)
  * loofah (2.19.1)
  * mail (2.8.0)
  * method_source (1.0.0)
  * middleware (0.1.0)
  * mime-types (3.4.1)
  * mime-types-data (3.2022.0105)
  * mini_magick (4.12.0)
  * mini_mime (1.1.2)
  * minitest (5.17.0)
  * multi_test (0.1.2)
  * net-imap (0.3.4)
  * net-pop (0.1.2)
  * net-protocol (0.2.1)
  * net-smtp (0.3.3)
  * nokogiri (1.13.10)
  * nori (2.6.0)
  * os (1.1.4)
  * page-object (2.3.1)
  * page_navigation (0.10)
  * pg (1.4.5)
  * protobuf-cucumber (3.10.8)
  * pry (0.13.1)
  * pry-awesome_print (9.6.11)
  * pry-byebug (3.9.0)
  * public_suffix (5.0.1)
  * racc (1.6.2)
  * rack (3.0.3)
  * rails-dom-testing (2.0.3)
  * rails-html-sanitizer (1.4.4)
  * rake (13.0.6)
  * regexp_parser (2.6.1)
  * rexml (3.2.5)
  * ruby-graphviz (1.2.5)
  * ruby-oci8 (2.2.12)
  * rubyzip (2.3.2)
  * savon (2.13.0)
  * selenium-webdriver (4.1.0)
  * sys-uname (1.2.2)
  * thor (1.2.1)
  * thread_safe (0.3.6)
  * timeout (0.3.1)
  * titleize (1.4.1)
  * tzinfo (2.0.5)
  * wannabe_bool (0.7.1)
  * wasabi (3.8.0)
  * watir (7.1.0)
  * watir-screenshot-stitch (0.8.0)
  * webdrivers (5.2.0)
  * webrick (1.7.0)
  * yard (0.9.28)
  * yard-cucumber (4.0.0 679d8a8)
  * yml_reader (0.7)
  * zeitwerk (2.6.6)
Use `bundle info` to print more detailed information about a gem

Ruby v2.6.0 Windows 11 Browsers tried - Chrome v108x, Edge, Firefox

Should I move this comment to html-formatter ?

[pgundlupetvenkatesh]

@mattwynne did you get a chance to look at my comment above?

[luke-hill]

Hi @pgundlupetvenkatesh the HTML reporter isn't globally broken.

If this is still an issue, could you create a new issue request with a repo that we can clone and produce a failing report as an MVCE.