rflechner / ScrapySharp

reborn of https://bitbucket.org/rflechner/scrapysharp
MIT License
346 stars 75 forks source link

Cookie error on web browser simulation example #22

Closed LGFN closed 4 years ago

LGFN commented 5 years ago

The basic web browser simulation example provided in the readme throws me 2 System.AggregateException inner exceptions. 1: CookieException: An error occurred when parsing the Cookie header for Uri 'http://www.bing.com/'. and 2: CookieException: The 'Path'='/search' part of the cookie is invalid.

For reverence, here's the code in its entirety:

public static void Test()
{
    ScrapingBrowser browser = new ScrapingBrowser();

    //set UseDefaultCookiesParser as false if a website returns invalid cookies format
    //browser.UseDefaultCookiesParser = false;

    WebPage homePage = browser.NavigateToPage(new Uri("http://www.bing.com/"));

    PageWebForm form = homePage.FindFormById("sb_form");
    form["q"] = "scrapysharp";
    form.Method = HttpVerb.Get;
    WebPage resultsPage = form.Submit();

    HtmlNode[] resultsLinks = resultsPage.Html.CssSelect("div.sb_tlst h3 a").ToArray();

    WebPage blogPage = resultsPage.FindLinks(By.Text("romcyber blog | Just another WordPress site")).Single().Click();
}
dararish commented 5 years ago

@LGFN isn't this the same issue as #1?

LGFN commented 5 years ago

Possible, I'm not well versed in the cookies tech to be able to determine similarities between these two issues. Actually, I'm not clear with the issue of #1 at all, nor do I understand its workaround. All I know is that I copied/pasted the basic example on the readme page and it did not work, and if there is some method of making it work, I believe it should be added to the readme.

khantoocool commented 4 years ago

@LGFN I came across the same issue and the only thing that helped me was bellow code line.

browser.IgnoreCookies = true;

LGFN commented 4 years ago

That did it, thank you!

Sumirmat97 commented 4 years ago

@LGFN I came across the same issue and the only thing that helped me was bellow code line.

browser.IgnoreCookies = true;

It worked thanks a lot