kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.27k stars 611 forks source link

Error with cookies file #979

Open mossmoss opened 1 year ago

mossmoss commented 1 year ago

I'm trying to run facebook-scraper on a Mac (Monterey) with homebrew installed python3 and pip3 - and pip3 installed facebook-scraper (0.2.59). I'm running into an issue I'm not sure how to resolve. I saved my cookies with the "Get cookies.txt" plugin in Chrome for facebook.com. Both c_user and xs are present. On a first save the domains listed are .facebook.com - later visits also have www.facebook.com entries. I tried with both cookie files and received the same error, in both cases it seems to be complaining about the first cookie, on line 5 - which is the datr cookie for .facebook.com (cookie obfuscated):

~/facebook via 🐘 via 🐍 v3.11.2 
❯ /usr/local/bin/python3 /usr/local/bin/facebook-scraper -c="/Users/user/Downloads/facebook.com_cookies.txt" --filename zuck.csv --pages 3 zuck
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/facebook_scraper/utils.py", line 225, in parse_cookie_file
    data = json.loads(data)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/facebook_scraper/utils.py", line 265, in parse_cookie_file
    domain, _, path, secure, expires, name, value = line.split('\t')
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: too many values to unpack (expected 7)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/facebook_scraper/__init__.py", line 441, in write_posts_to_csv
    for post in get_posts(
                ^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/facebook_scraper/__init__.py", line 196, in get_posts
    set_cookies(cookies)
  File "/usr/local/lib/python3.11/site-packages/facebook_scraper/__init__.py", line 40, in set_cookies
    cookies = parse_cookie_file(cookies)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/facebook_scraper/utils.py", line 267, in parse_cookie_file
    raise exceptions.InvalidCookies(f"Can't parse line {i + 1}: '{line}'")
facebook_scraper.exceptions.InvalidCookies: Can't parse line 5: '.facebook.com  TRUE    /   TRUE    1030849875  datr    XxC_6UIunU3ur80G    18'
Couldn't get any posts.

Thanks for any help!

PitBosch commented 1 year ago

I have the same error. I don't think that the problem is on the specific cookie cause by deleting it the error doesn't go away. Moreover I think that it wasn't a change previously made in this repository, since last month this worked and no major changes were made

mossmoss commented 1 year ago

@PitBosch Do you think it might be some newer python module? Maybe the latest Get cookies.txt changes the format (I doubt it)?

I have a WSL2/Ubuntu system where facebook-scraper 0.2.59 is working -

❯ pip3 freeze|grep -i json
demjson==2.2.4
demjson3==3.0.5
fastjsonschema==2.16.1
json5==0.9.10
jsonpatch==1.22
jsonpointer==2.0
jsonschema==3.2.0
simplejson==3.16.0

I have 2 Mac laptops where 0.2.59 doesn't work. I frequently brew update/upgrade - on these, there is only

❯ pip3 freeze|grep -i json
demjson3==3.0.6

Maybe some required modules are missing or are too new on the Mac? I believe I also installed the requirements.txt separately in debugging and it didn't help. I wasn't able to successfully install requirements-dev.txt (some error with yarl). I uninstalled facebook-scraper and reinstalled it without reinstalling requirements.txt separately.

pvgandhi404 commented 1 year ago

I had the same issue. Try using another extension like edit this cookie, copy the cookies and store them in a JSON file instead.

mossmoss commented 1 year ago

I had the same issue. Try using another extension like edit this cookie, copy the cookies and store them in a JSON file instead.

This worked for me on my previously broken Macs! I'm wondering why text format cookies are working on my other system and my Macs seem to want JSON cookies?

supremeconset commented 1 year ago

You need to disable cookie size value on plugin. image