pingora cache example

[MMADUs]

is there an example of pingora proxy cache implementation? thanks.

[Object905]

Here you go https://gist.github.com/Object905/cf10ffd97595887bb7b3868c89a793d7

Can ignore serde.rs, I use it to save/restore caches between restarts. Also compression.rs is WIP, but seems to work for me.

[Object905]

Also note that there is pingora::cache::MemCache ref https://github.com/cloudflare/pingora/issues/137

[MMADUs]

hey @Object905 really appreciate the code snippets. i have some few questions, how would i implement this in the ProxyHttp trait from pingora proxy? thanks.

[Object905]

I do it like so.

Create "cache" with lazy static with CacheBucket util from gist

pub static STATIC_CACHE: Lazy<CacheBucket> = Lazy::new(|| {
    use pingora::cache::lock::CacheLock;

    CacheBucket::new(
        SccMemoryCache::with_capacity(8192)
            .with_reject_empty_body(true)
            .with_max_file_size(Some(MB * 8))
            .with_compression(LZ4Compression),
    )
    .with_eviction(LRUEvictionManager::<16>::with_capacity(MB * 128, 8192))
    .with_cache_lock(CacheLock::new(Duration::from_millis(1000)))
});

Then in ProxyHttp ProxyHttp::request_cache_filter - session.cache.enable(...) (or CacheBucket.enable if you decide to use it) when request needs caching (should probably filter for only GET requests, but depends on your use case). ProxyHttp::cache_key_callback - maybe make a key for entry (by default its uri of the request) ProxyHttp::response_cache_filter - specify ttl of entry in CacheMeta

[MMADUs]

@Object905 i have some issues serialize and deserialize the bytes on BinaryMeta and the body bytes, do you know how i can fix this? the issue is on the scc_cache gist, i guess i need the full code base to understand the whole process especially how to set the cache data and get the cache data in proxyhttp trait, anyway thanks.

[Object905]

@MMADUs I've update gist with serde_example.rs and service_registry.rs. That's how I create the caches, so they're loaded on startup and saved periodically on background.

[MMADUs]

@Object905 i have an issue following your code. this happend on serialize and deserialize for the bytes::Bytes. do you got any idea how i can solve this?

[Object905]

Seems like you have to enable serde feature on bytes crate. Or refactor and just use Vec<u8> instead.

My related Cargo.toml

scc = { version = "2.0.19", features = ["serde"] }
bincode = { version = "=2.0.0-rc.3", features = ["serde"] }
bytes = { version = "1.7.0", features = ["serde"] }
serde = { version = "1.0.203", features = ["derive", "rc"] }
serde_json = "1.0.115"

[MMADUs]

hey @Object905 i got everything working! thank you very much for helping!!!

[MMADUs]

hey @Object905 i was wondering about your caching implementation, how did you manage to handle or revalidate the expired cachemeta? in my case when the cachemeta expires, the data in the hashmap doesnt automatically purged, and that way i keep inserting new upstream response. is there anything i can do to automate the revalidation when cachemeta expires? thanks