Right now we have a random wait time for all sites, so if a twitter follows a Facebook the wait is the same as if a twitter follows a twitter. Instead we should do elapsed time since the last scrape of the same type.
We should also set up pools of queues so Hypatia or Grigori can pull a scrape from a different type who's wait time has elapsed, allowing for more constant scraping.
Right now we have a random wait time for all sites, so if a twitter follows a Facebook the wait is the same as if a twitter follows a twitter. Instead we should do elapsed time since the last scrape of the same type.
We should also set up pools of queues so Hypatia or Grigori can pull a scrape from a different type who's wait time has elapsed, allowing for more constant scraping.