wummel / linkchecker

check links in web documents or full websites
http://wummel.github.io/linkchecker/
GNU General Public License v2.0
1.42k stars 234 forks source link

LinkChecker sends cookies in a different way from browsers #294

Closed wummel closed 11 years ago

wummel commented 11 years ago

Converted from SourceForge issue 3346972, submitted by karen-chan

When I use LinkChecker with cookies, it sends one cookie per header, which is different from browsers like Firefox and Chrome, they send all cookies in one header.

Here is an example cookie file: Host: localhost Set-Cookie: cookie1="value1" Set-Cookie: cookie2="value2" Set-Cookie: cookie3="value3"

which I called linkchecker.cookies

I invoked linkchecker like this: linkchecker --cookies --cookiefile linkchecker.cookies http://localhost/

LinkChecker sends the cookies like this: Cookie: cookie1=value1; $Path="/"; $Domain=localhost\r\n Cookie: cookie2=value2; $Path="/"; $Domain=localhost\r\n Cookie: cookie3=value3; $Path="/"; $Domain=localhost\r\n

Firefox sent the same cookies like this: Cookie: cookie1=value1; cookie2=value2; cookie3=value3\r\n

This is a big problem for me because my server software only uses the first "Cookie:" line, so the server sees the cookie as: {"cookie1": "value1", "$Path": "/", "$Domain": "localhost"}

Would it be possible to change this in linkchecker so it sends cookies like firefox please?

Here's my patch for linkchecker to send cookies in one header:

diff --git a/linkcheck/checker/httpurl.py b/linkcheck/checker/httpurl.py index e674f8f..6c9e681 100644 --- a/linkcheck/checker/httpurl.py +++ b/linkcheck/checker/httpurl.py @@ -552,10 +552,12 @@ class HttpUrl (internpaturl.InternPatternUrl, proxysupport.ProxySupport): host, port = urllib.splitnport(host, port) path = self.urlparts[2] self.cookies = self.aggregate.cookies.get(scheme, host, port, path)

wummel commented 11 years ago

Submitted by calvin

Will look into this. Must be fixed together with #3375899.

wummel commented 11 years ago

Submitted by calvin

We believe that the issue you reported is fixed in the source repository of linkchecker which can be found under: https://linkchecker.git.sourceforge.net/

Changelog entry:

Thank you for reporting the issue. It is now marked as pending and will be closed automatically in two weeks. If you believe that the issue is not fixed appropriately just add a comment to this ticket. This will set the status from "pending" to "open" again.

(This message was generated automatically.)