Comment scraper HTTP Error 400 for statuses with many comments

[AakaashRao]

When scraping comments from statuses with a large number of comments, the Graph API will begin returning a 400 error after a few thousand comments. I've seen this post, but the error often occurs substantially before the 25,000th comment. Does anyone have any suggestions?

To reproduce:

https://graph.facebook.com/v2.9/95475020353_10159243063820354/comments/?limit=100&access_token=[YOUR APP ID]|[YOUR APP SECRET]&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=&after=NTIZD&fields=id,message,reactions.limit(0).summary(true),created_time,comments,from,attachment

[datavistics]

Im also having this issue. It seems sporadic.. @minimaxir do you have any ideas? Ive gotten 2 issue types so far:

1. Empty data

2. {
   "error": {
     "message": "Unsupported get request. Object with ID '826223557471634_1610792979014684' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api",
     "type": "GraphMethodException",
     "code": 100,
     "error_subcode": 33,
     "fbtrace_id": "CFpGH0QT0jX"
   }
   }

I think a skip option would be helpful after a certain amount of attempts? Increasing the timeout helps with some cases. Thanks