KeithCu / LinuxReport

Customizable Linux news site based on Python / Flask
https://covidreport.keithcu.com/
GNU Lesser General Public License v3.0
10 stars 2 forks source link

Prevent excess RSS requests #10

Closed KeithCu closed 4 years ago

KeithCu commented 4 years ago

When a feed has expired, if two people come to the website at the same time, it will possibly fetch for both of them. Implement some logic which if the feed has expired: Check for URL + "FETCH" in the cache if it doesn't exist, then create an entry containing this PID / TID Then check to make sure it's our PID / TID

If it already exists and it's not our PID / TID, sleep for 100 ms and keep checking till cache entry disappears If it is, then fetch the RSS feed, add to cache, and then delete the URL + FETCH cache entry