minimaxir / facebook-page-post-scraper

Data scraper for Facebook Pages, and also code accompanying the blog post How to Scrape Data From Facebook Page Posts for Statistical Analysis
2.12k stars 663 forks source link

Bugfix for missing comments #105

Open TGWesper opened 6 years ago

TGWesper commented 6 years ago

When scraping multiple posts in a row, any post that is scraped after a post with paged comments had a malformed base_url, containing multiple "&after=" parameters. Because of this, apparently only the last page of comments was scraped for all subsequent posts. Resetting the "after" variable at the start of the loop fixes this. In the for-loop for subcomments this was already implemented correctly.