kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.45k stars 633 forks source link

Cannot seem to access anything on mobile url via scraper #756

Open kapitanluffy opened 2 years ago

kapitanluffy commented 2 years ago

So I tried accessing only the home page. Facebook seems to be detecting that I am using a scraper and blocks it right away

Here's my modified code on __init__.py on line 395 (facebook_scraper.py is also modified to return the response immediately)

    testResponse = _scraper.get("https://m.facebook.com")
    print(testResponse.html.find("title", first=True).text) # <--- Title is "Error"
    return

Here's the preview of the html response image

Here's the trace:

Traceback (most recent call last):
  File "D:\bobs\facebook-scraper\facebook_scraper\__init__.py", line 443, in write_posts_to_csv
    for post in get_posts(
  File "D:\bobs\facebook-scraper\facebook_scraper\__init__.py", line 220, in get_posts
    _scraper.login(*credentials)
  File "D:\bobs\facebook-scraper\facebook_scraper\facebook_scraper.py", line 916, in login
    response = self.get(self.base_url)
  File "D:\bobs\facebook-scraper\facebook_scraper\facebook_scraper.py", line 877, in get
    raise exceptions.UnexpectedResponse("Your request couldn't be processed")
facebook_scraper.exceptions.UnexpectedResponse: Your request couldn't be processed
Couldn't get any posts.

Take note that www.facebook.com seems to be working properly

kapitanluffy commented 2 years ago

I am using windows btw.

neon-ninja commented 2 years ago

Try cookies instead of credentials

kapitanluffy commented 2 years ago

I also tried that. Seems to be a problem on windows. I used ubuntu and it seemed to work.