speced / respec

A tool for creating technical documents and web standards
https://respec.org/
Other
709 stars 376 forks source link

stringifyJSHandle hides certain error details when run from command line #4736

Open msporny opened 3 weeks ago

msporny commented 3 weeks ago

Description of problem

Bare with me, this one is difficult to explain. More recently (ReSpec v35.1.0 -- June 2024), when ReSpec is invoked via the command line, some ReSpec processing errors result in [ERROR] undefined being reported to the console. This causes Github Actions to fail without providing a useful error. When the same file is processed through a graphic Chromium session, depending on the complexity of the ReSpec document, sometimes a more detailed error is reported, and sometimes no error is reported at all.

URL to affected spec or repo:

https://github.com/speced/respec/blob/main/tools/respecDocWriter.js#L320

What happened (e.g., it crashed)?:

Given this basic ReSpec HTML document as input (only the citation for [[[PRIVACY-BY-DESIGN]]] matters, because processing it is what throws the error):

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>ReSpec citation bug v1.0</title>
    <script
      src="https://www.w3.org/Tools/respec/respec-w3c"
      class="remove"
      defer
    ></script>
    <script class="remove">
      // All config options at https://respec.org/docs/
      var respecConfig = {
        specStatus: "ED",
        editors: [{ name: "Your Name", url: "https://your-site.com" }],
        github: "some-org/mySpec",
        shortName: "respec-citation-bug",
        xref: "web-platform",
        group: "vc",
      };
    </script>
  </head>
  <body>
    <section id="abstract">
      <p>This is required.</p>
    </section>
    <section id="sotd">
      <p>This is required.</p>
    </section>
    <section class="informative">
      <h1>Demonstration of Bug</h1>
      <p>The bug appears when you cite [[[PRIVACY-BY-DESIGN]]].</p>
    </section>
    <section id="conformance">
      <p>
        This is required for specifications that contain normative material.
      </p>
    </section>
  </body>
</html>

Processing the document above leads to this error in a Chromium Version 121.0.6167.139 browser:

image

and this is what is reported when the same file is processed using the respec command line (via npx respec -s https://website.example/respec-bug.html -o index.html.built.html --verbose -t 60 -e):

image

and this is what it looks like in a Github Action failure:

image

I spent a bit of time and was able to track this down to the stringifyJSHandle call here:

https://github.com/speced/respec/blob/main/tools/respecDocWriter.js#L320

... which is dealing with a message that has this form:

{type, msgText, text, args} -> { "error", "JSHandle@error", "{}", ['{}'] }

... and that results in the [ERROR] undefined text being printed to the console when respec is invoked as a command line tool. The bug is triggered because the specref entry for [[[PRIVACY-BY-DESIGN]]] seems to be malformed (that's a separate problem I'll chase down on the specref repo). However, the browser console shows the error (sometimes), but the command line seems to swallow the error.

Expected behavior (e.g., it shouldn't crash):

The command line error should result in something that's more descriptive than [ERROR] undefined.

I did put in additional debug code and restructured the code a bit over an hour of trying to figure out where the issue was happening in ReSpec, and every now and then I could get a more detailed stack trace to appear, but it was sporadic (signalling some possible async issue) and I was trying out a few different debug approaches so I hesitate to guess at what is happening here.

Optional, steps to reproduce:

  1. Use the HTML file provided above.
  2. Run some variation of npx respec -s https://website.example/respec-bug.html -o index.html.built.html --verbose -t 60 -e on it.
  3. You should see [ERROR] undefined printed out to the console with a not very useful stack trace.

Breakpoint'ing or adding debugging statements around https://github.com/speced/respec/blob/main/tools/respecDocWriter.js#L320 might help.

/cc @iherman

sidvishnoi commented 3 weeks ago

Thank you for the detailed error report! I'll make sure ReSpec doesn't swallow details 😉 Hopefully this weekend..