search

ekkyarmandi / text-scraping

HTML text scraping
1 stars 2 forks source link

Tried the cookie feature on github.com, got this error message. #3

Open why-not opened 2 years ago

why-not commented 2 years ago

Traceback (most recent call last): File "/Users/username/Desktop/Dropbox/Parent/workspace/project_name/machine_learning/text-scraping-master/app.py", line 14, in crawler.run() File "/Users/username/Desktop/Dropbox/Parent/workspace/project_name/machine_learning/text-scraping-master/crawler.py", line 35, in run self.add_cookies() File "/Users/username/Desktop/Dropbox/Parent/workspace/project_name/machine_learning/text-scraping-master/crawler.py", line 77, in add_cookies self.browser.add_cookie(dict( File "/Users/username/Tools/anaconda3/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 1136, in add_cookie self.execute(Command.ADD_COOKIE, {'cookie': cookie_dict}) File "/Users/username/Tools/anaconda3/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute self.error_handler.check_response(response) File "/Users/username/Tools/anaconda3/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.UnableToSetCookieException: Message: unable to set cookie

(Session info: headless chrome=102.0.5005.61)

why-not commented 2 years ago

I installed the exact chromedriver-binary as above (102.0.5005.61) using

pip install chromedriver-binary==102.0.5005.61.0

But I still get the same error as "Unable to set cookie"

ekkyarmandi commented 2 years ago

It seems like not the driver issue. It works really well on my machine

from crawler import WordsCrawler

url = "https://www.facebook.com/groups/697763397093911"
cookies = "data/www.facebook.com_04-06-2022.json"
output_name = "facebook"

crawler = WordsCrawler(
    url=url,
    cookies_path=cookies,
    output_path=output_name,
    headless=False
)

crawler.run()
why-not commented 2 years ago

ekky,

I tried cookie feature with the GitHub.com website.

I created this test account for us.

username: wtozmopsfnnxwtiryg@nthrl.com password: 9&9&9&9&9&a

Once you are logged in export the cookies using the extension https://chrome.google.com/webstore/detail/j2team-cookies/okpidcojinmlaakglciglbpcpajaibco/reviews

If not you can also use the cookie that I have attached for your convenience with this message. https://gist.githubusercontent.com/why-not/986fc89385e458f64d6142abb91aa5ab/raw/36a20befe04658cb3dcc7f9240fa456349d0e38e/github_cookie_test.json

It prints the above "Unable to set cookie" message.

Let me know,

thank you,