apache / incubator-pagespeed-ngx

Automatic PageSpeed optimization module for Nginx
http://ngxpagespeed.com/
Apache License 2.0
4.37k stars 363 forks source link

Fully memory cache #182

Closed Profforgr closed 11 years ago

Profforgr commented 11 years ago

Hello.

I assume that ngx_pagespeed + memcached will only cache results from reading files under cache directory. Can 'cache directory' be fully removed - so can we cache fully in memory?

Also it would be really perfect to make Nginx's cache, not memcached. Memcached sucks, it slow's down's the whole structure and if i'll add ngx_pagespeed + memcached to my server - i'll see slow down and i'll have no choice expect to remove ngx_pagespeed and memcached. I can not cache in the disk, because it 1) Wastes free space 2) Load my hard-drive cause it to be overloaded and slow down's parralel requests 3) I just enjoy no-delay things (so RAM cache would be much faster).

So i want to apply such addon to ngx_pagespeed.

We store only 100000 files in RAM and if it goes high - we clean files via LRU (Least

Requested URI)

If the file was not requested in 12h - then we remove it from our cache. I set such time

because we should rely on the Time Zones. If the file is not popular for 12h - it's OK, but # if more - then it is not a really popular files. Possibly someone request this file too many # times.

pagespeed_file_cache max=100000 inactive=12h;

time in minutes or 'until it would be changed' which can be shortened to 'until_changed'

pagespeed_file_cache_valid until_changed;

We would like to cache only popular files. Need to be used 10 times in 12 hours

pagespeed_file_cache_min_uses 10;

On or Off pagespeed file caching in memory

pagespeed_file_cache_state on;

jeffkaufman commented 11 years ago

If you don't want to use a disk cache with ngx_pagespeed there are currently two ways to do that:

1) memcached 2) file cache on tmpfs (ramdisk)

If you're only on a single server then (2) is probably best; if you want to share your cache between servers then you need (1).

You mention wanting support for cache eviction based on "LRU (Least Requested URI)", which is confusing. LRU is an acronym for "Least Recently Used" and it sounds like what you actually want is LFU "Least Frequently Used". All our caches evict the entry that has gone the longest without access, which is a "Least Recently Used" policy.

Why do you want to limit based on a number of files instead of total size? Regardless, if you use the file cache on tmpfs approach you can specify both limits:

pagespeed FileCacheSizeKb 102400;
pagespeed FileCacheInodeLimit 500000;

You also ask for:

# time in minutes or 'until it would be changed' which can be shortened to 'until_changed'
pagespeed_file_cache_valid until_changed;

Could you explain why you need this? If pagespeed rewrites an image to make it smaller, the source image isn't changed on disk, and it's loaded reasonably often, why would you want to remove it from cache?