search

tokee / lucene-solr

High cardinality faceting (SOLR-5894)
http://tokee.github.io/lucene-solr/
7 stars 1 forks source link

Reduce upstart memory requirements #29

Closed tokee closed 9 years ago

tokee commented 9 years ago

Practically all sparse counters require knowledge of the counter distribution. This is derived by iterating the Solr doc->ordinal structures and collecting statistics in a temporary large counter structure (int[#unique values]). In the special case of a fully optimized segment, this information can be derived on a streaming basis, thus eliminating the need of the temporary large stat collector.

tokee commented 9 years ago

This is only true for String fields with DocValues.