search

wjdp / htmltest

:white_check_mark: Test generated HTML for problems
MIT License
323 stars 54 forks source link

External link check fails with htmltest but works fine with cURL and in browser #214

Closed Simran-B closed 1 year ago

Simran-B commented 1 year ago

Describe the bug

There is a specific external link in the content I run htmltest over that keeps getting reported as broken. However, if I try a tool like cURL or simply open the link in my browser, it works flawlessly.

To Reproduce

  1. Create a test file page.html:

    <!DOCTYPE html>
<html>
   <head>
       <title>Page</title>
   </head>
   <body>
       <p>
           Link: <a href="https://www.itu.int/en/Pages/default.aspx">ITU</a>
       </p>
   </body>
</html>

  2. Run htmltest for this file with the default options (CheckExternal: true and ExternalTimeout: 15)

  3. There is seemingly a high chance that you get a dial tcp error:

    • Get "https://www.itu.int/en/Pages/default.aspx": dial tcp: lookup www.itu.int on 192.168.65.7:53: read udp 172.17.0.3:40610->192.168.65.7:53: i/o timeout --- docs/3.10/arangograph/certificates.html --> https://www.itu.int/en/Pages/default.aspx (reported for my actual setup, with ExternalTimeout: 60; why is UDP involved?)
    • Get "https://www.itu.int/en/Pages/default.aspx": dial tcp: lookup www.itu.int: no such host --- page.html --> https://www.itu.int/en/Pages/default.aspx (reported for the above test file)

Expected behavior

The link should be reported as working if it actually works.

Actual behavior

I got errors like the above multiple times with htmltest but never with other programs. It eventually did report the link with HTTP status code 200 using the test file after a retry, however.

Versions

wjdp commented 1 year ago

Cannot verify this issue. I'm getting pass when trying the above.

From the log line, UDP and port 53 on 192.168.65.7 (private DNS server) this looks to be a DNS lookup issue likely specific to your network/environment.

Simran-B commented 1 year ago

I see. Thanks for checking. I guess I was lucky with other tools and unlucky with htmltest.