Closed tobi007 closed 5 years ago
I've been messing with this for a few hours. I've been able to get it to work, if I disable the redirect after login. I don't think the redirect specifically is causing the problem...trying to dig down further.
Basically, I added a c.RedirectHandler() that just returns http.ErrUseLastResponse. Also, I ignore the error from the client.Post(loginURL,data) as it propogates that error. After that, the Visit to the projectsURL goes through without an issue...but I think I'm missing the real issue here
Found the issue in the source code. The checkRedirectFunc()
by default copies the request headers from the previous request. In this case, theCookie
from the lastRequest
was no longer valid as it was replaced by the response from signin. Sending this old Cookie
invalidates the session and automatically logs you out of the system.
As a work around, you can override the RedirectHandler()
so that it just returns nil
, so it will not modify the current request. I will submit a PR to address this issue.
Thanks for the report!
please I trying to learn webscrapping using goquery and colly. The above code is from http://sandipbgt.com/2018/08/23/scraping-tutorial-with-golang/ for getting list of personal projects. The above code works, but I'm trying to do the same using colly but it's not working, I can't even get it to log in. Find the code below when I used colly.
This is the code trying it with colly. Please what am I missing.