Scale sample factor with result hit count for heuristic faceting

[tokee]

Currently heuristic faceting is controlled by a hitCount based limit facet.sparse.heuristic.fraction and a maxDocuments based facet.sparse.heuristic.sample.size.

The constant sample size means that the chance of errors with low hitCounts is high and the performance overhead with large hitCounts is unnecessarily high. Using a hitCounts based function for the sample size instead of a maxDocuments would remedy this.

If h is hitCount, d is maxDocuments and a & b are empirically determined constants, the formula a*(h/d)+c provides an output linear to hitCount over maxDocuments. Sanity checking by turning off heuristics when the sample size is > 50% and forcing a sample size >= 1‰ seems prudent.

Conservative: d = 100M, a = -1.0 and b = 1.0.

• h = 1M ⟹ -1.0*(1M/100M)+1.0 = 99%` ⟹ full count
• h = 10M ⟹ -1.0*(10M/100M)+1.0 = 10%` ⟹ full count
• h = 20M ⟹ -1.0*(20M/100M)+1.0 = 20%` ⟹ full count
• h = 50M ⟹ -1.0*(50M/100M)+1.0 = 50%` ⟹ full count
• h = 75M ⟹ -1.0*(75M/100M)+1.0 = 75%`
• h = 100M ⟹ -1.0*(100M/100M)+1.0 = 0%` ⟹ 1‰

Aggressive: d = 100M, a = -1.0 and b = 0.5.

• h = 1M ⟹ 49%
• h = 10M ⟹ 40%
• h = 20M ⟹ 30%
• h = 50M ⟹ 0% ⟹ 1‰
• h = 75M ⟹ -25% ⟹ 1‰
• h = 100M ⟹ -50% ⟹ 1‰

Empiric based example: d = 250M, a = -19 and b = 0.78.

• h = 1M ⟹ 70% ⟹ full count
• h = 10M ⟹ 2%
• h = 20M ⟹ -74%
• h = 50M ⟹ -302% ⟹ 1‰
• h = 75M ⟹ -492% ⟹ 1‰
• h = 100M ⟹ -682% ⟹ 1‰

[tokee]

The parameters for heuristics now makes it possible to scale sampling with both hitCount and maxSize, with three different strategies for sampling.