snowplow / enrich

Snowplow Enrichment jobs and library
https://snowplowanalytics.com
Other
21 stars 39 forks source link

Stream FS2: add cache metrics #376

Open chuwy opened 4 years ago

chuwy commented 4 years ago

In many IO-heavy enrichments we use cache (LRU mostly):

I'm wondering how many cache misses do we usually have, how much the caches used in general. Knowing that we can decide if we should execute those enrichments on separate thread pool (if still IO-heavy - separate blocking thread pool, if cache used up to 99% - main fixed thread pool).

chuwy commented 4 years ago

https://guava.dev/releases/22.0/api/docs/com/google/common/cache/CacheStats.html might be useful, given that scala-lru-map is a thin facade over Guava now.