search

4pr0n / gonewilder

GNU General Public License v2.0
32 stars 11 forks source link

Smarter polling [long-term issue] #34

Open wentwild opened 10 years ago

wentwild commented 10 years ago

I currently have around 1400 users that my installation is polling for content. It takes about 30 minutes to get through the list. Many of the accounts have not had any updates in a long time but the still get polled just as frequently as users who have recent submissions. In the long run this is going to be a big problem and will most likely result in missing out on content.

One thing that has occured to me is to create a group of users who have been inactive for X time period. The process could poll some subset of this group after grabbing the new top content week. Then go through the list of active users again like it currently does.

4pr0n commented 10 years ago

One option is to use the "friends" functionality (--just-friends), which only polls /r/friends for updates every 2 seconds.

It should still work...

4pr0n commented 10 years ago

Then again, the last-updated time for each user is available in the database... Coming up with an algorithm that checks new accounts quickly and occassionally checks older accounts would require some thought.