wummel / linkchecker

check links in web documents or full websites
http://wummel.github.io/linkchecker/
GNU General Public License v2.0
1.42k stars 234 forks source link

Not working with ASP.Net forms authentication #436

Open mwittmann opened 10 years ago

mwittmann commented 10 years ago

I'm trying to use LinkChecker on ASP.Net MVC3 web application I've built that use ASP.Net forms authentication. I have configured the LinkChecker config file to correctly log in to the login page:

    entry=.* mwittmann@mydomain.com
    loginurl=http://localhost/MyWebApp/Account/LogOn
    loginuserfield=username
    loginpasswordfield=password

When I run LinkChecker, I am prompted for the password, and using Fiddler, I see that the login is successful and that the two "Set-Cookie" headers required by ASP.Net forms authentication are returned to the browser:

    Set-Cookie: ASP.NET_SessionId=dnf5ghbfpmr0bauizwhqvn5n; path=/; HttpOnly
    Set-Cookie: MYFORMAUTH=6E56CA...cookie_value_is_about350_characters_long...8D3AF1B3; path=/; HttpOnly

The login page also sends a 302 redirect to a new URL which, via Fiddler, I can see is requested with the above HTTP cookies and succeeds.

It appears that the initial authentication above is done via the Twill library, because in Fiddler I see the user agent looks like IE6:

    User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)

However subsequent LinkChecker requests send the user agent as:

    User-Agent: Mozilla/5.0 (compatible; LinkChecker/8.4; +http://wummel.github.com/linkchecker/)

And the subsequent requests do not include the two HTTP cookies above. In fact, the only cookies sent are:

    Authorization: Basic bWFydGluQGF0c3dvcmxkd2lkZS5jb206YWJjZDEyMw==
    DNT: 1

Since the ASP.Net forms authentication cookies are not included, my web app redirects back to the login page, so LinkChecker cannot access any of the URLs that require authentication.

I am surprised that there are no issues logged for ASP.Net forms authentication and these cookie values, so I wonder if I am misunderstanding something basic. Unfortunately, since I do not (yet) know Python, I don't want to try to digging into the code myself.

nicholassmeaton commented 9 years ago

I can report that this issue is still present in version 9.3.