Caching middleware

[ksysoev]

It could be common feature request to cache read queries for many apis. we should design and implement middleware that will be caching responses from BE and sending them back to the client

[shan-96]

So are you saying the this will be a server side cache?

[ksysoev]

@shan-96 yes the idea to have some thing similar to rate limiter middleware, https://github.com/ksysoev/wasabi/blob/main/middleware/request/ratelimiter.go#L17... to accept cache key generator from the user and only focus on caching logic. For now we can keep implementations simple and use some in memory cache, try to find some lru cache with ttl support. for handling concurrent request to the same key we can use https://pkg.go.dev/golang.org/x/sync/singleflight

[ksysoev]

also for catching api response we can use connection wrapper https://github.com/ksysoev/wasabi/blob/main/channel/connection_wrapper.go

[shan-96]

I'm adding some proposals here. Let me know if a document with details are needed

Caching lib prospects

• https://github.com/VictoriaMetrics/fastcache - thread safe + in memory, might be suitable for large number of entries and small GC overhead
• https://github.com/dgraph-io/ristretto - probably same as above but more popular and there might be option to offload the cache to separate layer or outside main memory
• https://github.com/akyoto/cach - very simple, does one job
• https://github.com/OrlovEvgeny/go-mcache - pointer based in memory cache
• https://github.com/rocketlaunchr/remember-go - it aims to provides universal caching interface with support for redis, memcache and ristretto

I personally favor remember-go just because it is aiming to provide a caching interface rather than a caching library. However I also would like to keep things simple and use the most simple cache and write custom interfaces for users to implement themselves. We can also stick to single implementation rather than providing options and go with ristretto.

Would like to know what direction you want to go with

[ksysoev]

@shan-96 ristretto looks good, we can start with it, as a next steps we can provide option to provide customer cache storage that satisfy the interface.

[ksysoev]

@shan-96 do you have any concern if i takeover this issue?

[shan-96]

@ksysoev go ahead

[ksysoev]

Implemented in #97, I decided to go with https://github.com/jellydator/ttlcache as it looks way simpler and looks enough for current needs.

[shan-96]

ah yes ttlcache is a good alternative