kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.27k stars 609 forks source link

Unable to extract all comments and replies of the facebook post #869

Open ajitkshirsagar opened 1 year ago

ajitkshirsagar commented 1 year ago

While running the scrapper to extract post, comments, and reactions, I am getting the following exceptions:

 File ".pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/utils.py", line 279, in safe_consume
    for item in generator:
  File ".pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/extractors.py", line 1140, in extract_comment_replies
    for action in data['payload']['actions']:
TypeError: 'NoneType' object is not subscriptable
Exception when consuming <itertools.chain object at 0x109141f40>: <class 'TypeError'>: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/.pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/utils.py", line 279, in safe_consume
    for item in generator:
  File "/.pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/extractors.py", line 1140, in extract_comment_replies
    for action in data['payload']['actions']:
TypeError: 'NoneType' object is not subscriptable
Exception when consuming <itertools.chain object at 0x109176760>: <class 'TypeError'>: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/.pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/utils.py", line 279, in safe_consume
    for item in generator:
  File "/.pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/extractors.py", line 1140, in extract_comment_replies
    for action in data['payload']['actions']:
TypeError: 'NoneType' object is not subscriptable

I am running the scrapper using the following:

facebook-scraper --filename fb-data-part0.json --timeout 10000 --comments --reactions --reactors --format=json --days-limit 70 --verbose --cookies cookies.txt --resume-file resume.txt anwb
pribadihcr commented 9 months ago

@ajitkshirsagar did you solve it?

ajitkshirsagar commented 8 months ago

@ajitkshirsagar did you solve it?

@pribadihcr I managed to scrap some data with an actual user log-in session but after some time Facebook also found this activity and kicked me out.