gchq / stroom

Stroom is a highly scalable data storage, processing and analysis platform.
https://gchq.github.io/stroom-docs/
Apache License 2.0
431 stars 55 forks source link

Refactor caching of feeds/types in Meta(Feed|Type)?DaoImpl #3201

Closed at055612 closed 1 year ago

at055612 commented 1 year ago

MetaDaoImpl has a noddy hashmap based cache of feeds and types. This duplicates the caching done by MetaFeedDaoImpl and MetaTypeDaoImpl. MDI should make use of the caches in MFDI and MTDI rather than duplicating.

Also MFDI and MTDI should really support wild-carding (though not cache wild-carded keys).

There is a comment about not wanting threads to block in MDI, so need to be mindful of that if using caffeine, which would block if used as is. Might need to do getIfPresent on the cache and if not found then use the db to get a value synchronously then fire a async thread to load it in slow time.

Another odditity is why the maps in MDI have lists as the values. Feeds/types are unique and wild-carded keys cannot be cached so there should only be 0-1 IDs for each key.

at055612 commented 1 year ago

Fixed in > 7.0.23