search

AlJohri / socialscraper

Social media scrapers
25 stars 3 forks source link

Record cursor and postdata while performing graph search #4

Closed AlJohri closed 10 years ago

AlJohri commented 10 years ago

Although the search no longer gives errors, it gives a vastly different number of results each time, constantly decreasing. Seems like it went from 15k to 10k to 5k.

Figure out ways to deal with this -- perhaps by slowing it down a bit or switching accounts and taking the union of the results etc.

AlJohri commented 10 years ago

Reading this issue 3 weeks later again, like I said above, this could be a really good way to make the graph search more consistent.

But at the same time, even if you can rescrape a set of results by following the cursor history, what does this really accomplish? You've already scraped the data once, just back it up.

Closing this for now - not 'lean'.