OffHeap Lookup cache is broken (keeps everything off-heap AND on-heap)

[leventov]

Here: https://github.com/druid-io/druid/blob/78159d7ca442cf8d99ea356d39932ab3adb8e10f/extensions-core/lookups-cached-global/src/main/java/io/druid/server/lookup/namespace/cache/OffHeapNamespaceExtractionCacheManager.java#L79 it was supposed that MapDB caches 10 millions of bytes (see https://github.com/druid-io/druid/blob/420358029061acb09e5f40370881f88e5a36d8d5/docs/content/development/extensions-core/lookups-cached-global.md, search "10MB" in page), but actually MapDB caches 10 millions entries (check the MapDB source code).

As of now the recommendation should probably be just not to use off-heap lookup cache, until a suitable solution is found.

[b-slim]

@leventov i get the mismatch between the size/count of entries but not sure what you mean by both onheap and offheap ?

[leventov]

@b-slim as it is configured to cache that many entries it effectively caches every entry. So it keeps all entries on-heap (in cache) and off-heap.

[leventov]

@jon-wei could you add to Release Notes that off-heap QTL cache shouldn't be used?

[jon-wei]

@leventov added an item to the release notes linking to this issue

[leventov]

@jon-wei I would formulate it harder, like "must not be used because it is broken", but up to you

[jon-wei]

@leventov sure, I made the warning in the notes stronger

[leventov]

@jon-wei thanks