XSScrapy is really a great tool, however, I am struggling to crawl login areas. I try to provide the sessions via header and cookies, but it does only crawl the areas outside the login page (public accessible pages).
Is there an example how to run the tool with custom headers which include the user session?
XSScrapy is really a great tool, however, I am struggling to crawl login areas. I try to provide the sessions via header and cookies, but it does only crawl the areas outside the login page (public accessible pages). Is there an example how to run the tool with custom headers which include the user session?