search

apache / lucene

Apache Lucene open-source search software
https://lucene.apache.org/
Apache License 2.0
2.45k stars 975 forks source link

Suggestion about LRUQueryCache Optimization #13318

Closed boicehuang closed 1 month ago

boicehuang commented 2 months ago

Description

https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/LRUQueryCache.java#L747

Can we use ReentrantReadWriteLock in LRUQueryCache (Currently ReentrantLock) to allow reading cache concurrently, to improve search performance？

I benchmarked this optimization by mocking some random LongPoint and querying them with PointInSetQuery with bool filter.

doc count field cardinality query point baseline QPS candidate QPS diff percentage
30000000 10 1 2481 5102 105.6%
30000000 1000000 1 6396 6596.48 3.1%

I think this change can help filter queries that need to query low-cardinality fields. If it helps, I will submit a PR.

boicehuang commented 2 months ago

@jpountz what you think on this? Any help will be appreciated.

boicehuang commented 2 months ago

13306

jpountz commented 1 month ago

Fixed by #13306.