Closed stevecheckoway closed 1 year ago
I'm going to close this issue because I figured out that I, as the user, can configure htmlproofer using a cookie jar or not.
From the command line, it is
$ htmlproofer --typhoeus '{ "followlocation": true, "cookiefile": "cookiejar.txt", "cookiejar": "cookiejar.txt" }' /tmp/a.html
From Ruby, the configuration is something like
{
typhoeus: {
followlocation: true,
cookiefile: 'cookiejar.txt',
cookiejar: 'cookiejar.txt'
}
}
It may be worth adding this information to the configuration section of the README.
It may be worth adding this information to the configuration section of the README.
PRs accepted. 😀
Given the following document linking to a recent CNN tweet shows the problem.
Here's the input file.
Here's the output.
Here's the curl output.
If I had to guess, I'd say it's redirecting to the same location but setting cookies and the website is probably checking if the cookies are set. Indeed, with a little testing, this seems to be exactly what's happening.
If I configure
curl
to follow redirects (via-L
), I get an infinite loop. If I tellcurl
to use a cookie jar and follow the redirects, it succeeds.It seems like two approaches to dealing with this:
Location
header, treat the page as existing (although that won't work with hashes),