search

internetarchive / warcprox

WARC writing MITM HTTP/S proxy
378 stars 54 forks source link

Use in-memory LRU cache in CDX Server dedup #114

Closed vbanos closed 5 years ago

vbanos commented 5 years ago

Enable in-memory caching of CDX dedup requests using stdlib lru_cache method. LRU Cache size is 1024.

Cache memory info is available on INFO logging outputs like:


CacheInfo(hits=3172, misses=3293, maxsize=1024, currsize=1024)
``