project-lux / lux-marklogic

Code, issues, and resources related to LUX MarkLogic
Other
3 stars 2 forks source link

Explore means to support facets for broad searches (from 1176) #56

Open gigamorph opened 4 months ago

gigamorph commented 4 months ago

728 implemented a threshold that facet requests must meet else the request is rejected: the estimated number of search results multiplied by the number of values in the associated range index must be less than the threshold. That threshold is 5,000,000,000,000. This was implemented in response to #688 whereby calculating facets for broad searches consumed more system resources than worth populating facets that do not meet the subsequently-imposed threshold.

# In terms of estimated number of search results multiplied by the number of values in the facet's index,
# this is the highest multiplication product that the system will accept in facet requests. When the 
# product exceeds this value, the facet values will not be calculated.
#
# This threshold can and should be adjusted as we go.  It may be adjusted by modifying this a) property 
# value and deploying or b) directly editing the associated constant in the modules database *and* updating
# this property value.  The latter option avoids a deployment.  When updating the property value, decide if
# you should do so in gradle.properties or in an environment's properties file.
#
# The original threshold value was based on the associated bugherd's facet request, where:
# 
#   q=(works)
#   scope=work
#   facetNames=workCreationAgentId
#
# The estimated number of search results was 1093865 and the number of values in the workCreationAgentId
# index was 4924830.
facetMaximumProduct=5000000000000
roamye commented 2 months ago

Old tix: https://git.yale.edu/lux-its/marklogic/issues/1176

Is this still an issue? or can we close?

roamye commented 2 months ago

Prioritization Review as a Research Ticket. Discussed at ticket forming, noted as possibly future or very low priority.