Ignoring response 404 - getting empty csv

rugantio / fbcrawl

A Facebook crawler

Apache License 2.0

667 stars 229 forks source link

Open andsic12 opened 4 years ago

andsic12 commented 4 years ago

Hello, as per title, this is the error i receive, either when trying to scrape a page or comments:

[scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404 https://mbasic.facebook.com/login/device-based/regular/login/partitodemocratico/posts/2296031677301785>: HTTP status code is not handled or not allowed

I tried changing user agent, removing devices sessions as I read among the similar issues, nothing worked. Thanks in advance

ugonfor commented 3 years ago

Duplicate of #2 A simple workaround is to log in via your traditional web browser once and everything should work fine.