Closed tokee closed 9 years ago
The ground work had been implemented in the experimental pack
branch with the counters PackedOpportunistic
and NPlaneMutable
with the tank
option. Unfortunately, preliminary experiments shows that some of the DocValues structures are not thread safe and thus require an instantiation per thread. This means that there's an overhead to running extra threads, at least if one starts multiple threads per segment.
Most of the underlying Solr structures uses ThreadLocal so with a shared executor, the overhead of extra instantiations is small. The bitmap branch has a working implementation.
Recent experiments with
AtomicIntegerArray
andAtomicLongArray
suggests that they quite close to pureint[]
andlong[]
performance. Together with theadvance
method on result sets this opens up for threaded counter updates within single fields.This would mean a speed up of the counting phase roughly linear to free CPU power, for facets with a many (millions) of references from documents to values.