sohini2689 / gaewiki

Automatically exported from code.google.com/p/gaewiki
0 stars 0 forks source link

too many purge request cause free ratio reached very fast #87

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
What steps will reproduce the problem?
1. I have down load the new code from downloads and deploy the code

What is the expected output? What do you see instead?
The data read operation is very huge and easily to reach the free ratio.

What version of the product are you using? On what operating system?

82a99aa310d3

Please provide any additional information below.
----------------------

The problem should be the purge usage. According to the code, every time one 
page is edited, a purge request will be added to the queue. In the purge 
handler, there is a very dangerous iteration for all pages. This will cause 
many read operations. 

Class EditHandler():
.....
    taskqueue.add(url="/w/cache/purge", params={})

class CachePurgeHandler(webapp.RequestHandler):
    def get(self):
        if users.is_current_user_admin():
            taskqueue.add(url="/w/cache/purge", params={})

    def post(self):
        memcache.delete('Index:')
        memcache.delete('IndexFeed:')
        memcache.delete('Sitemap:')
        memcache.delete('Changes:')
        memcache.delete('ChangesFeed:')
        for page in model.WikiContent.all():
            memcache.delete('Page:' + page.title)
            memcache.delete('RawPage:' + page.title)
            memcache.delete('PageHistory:' + page.title)
            memcache.delete('BackLinks:' + page.title)
            for label in page.labels:
                memcache.delete('PagesFeed:' + label)
                memcache.delete('GeotaggedPagesFeed:' + label)
                memcache.delete('GeotaggedPagesJson:' + label)

--------------------------

I advise to remove the page cache update in the CachePurgeHandler. Instead, in 
the EditHandler, we can simple remove the cache of current edit page.

Class EditHandler():
.....
    def update_page_cache(page):
            memcache.delete('Page:' + page.title)
            memcache.delete('RawPage:' + page.title)
            memcache.delete('PageHistory:' + page.title)
            memcache.delete('BackLinks:' + page.title)
            for label in page.labels:
                memcache.delete('PagesFeed:' + label)
                memcache.delete('GeotaggedPagesFeed:' + label)
                memcache.delete('GeotaggedPagesJson:' + label)
    update_page_cache(page)

    #taskqueue.add(url="/w/cache/purge", params={})

Original issue reported on code.google.com by wangxing...@gmail.com on 18 Apr 2013 at 1:32