search

kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.47k stars 635 forks source link

Just started getting lots of UnexpectedResponse exceptions out of the blue #846

Open curiousier-george opened 2 years ago

curiousier-george commented 2 years ago

I'm wondering if anyone else might have just started getting

facebook_scraper.exceptions.UnexpectedResponse: Your request couldn't be processed

exceptions today or might know what's possibly causing it. My own thought is maybe either network errors or a temporary ban - or else Facebook just changed something that facebook-scraper doesn't know how to handle? Any thoughts or suggestions would be greatly appreciated!

curiousier-george commented 2 years ago

A little more info: I'm getting the exception while consuming the iterator returned by get_posts() in code like this:

posts = get_posts(username, options={'allow_extra_requests': False})
for post in posts:
    ...

It doesn't seem to be a temporary banning issue because if I immediately run the code again it works for a while, sometimes all the way through, until I get the exception again.

Barring a complete resolution, is there a way to handle the exception and just start consuming the iterator again, starting where it left off? I'm not sure how that would work since the exception is happening in an iterator and in the "for" part of a for loop.

neon-ninja commented 2 years ago

Resuming extraction after catching an exception is possible, see https://github.com/kevinzg/facebook-scraper/issues/310#issuecomment-852652846