Investigate 5xx caused by long search queries

[jcateswellcome]

Periodically we receive platform alerts for 5xx caused by searches for long titles. Before we decide on how we can minimise or stop this issue, it would be useful to have some more insight into current usage patterns and limitations.

• [ ] determine how frequently this problem occurs
• [ ] explore and understand the character or token limits that trigger 5xx
• [ ] analysis of query length so that we better understand user behaviour
• [ ] what is the behaviour on comparable sites when using one of the long titles

More background on this can be found here, with examples of searches that trigger errors: https://wellcome.slack.com/archives/C3TQSF63C/p1713776447397219

[jcateswellcome]

Relate to epic: https://www.notion.so/wellcometrust/Search-relevance-improvements-2ea0f44a0e6e44b38fd3ec6e587b23c4?pvs=4

[pollecuttn]

Investigate not fully analysing queries past a certain number of tokens (number tbd)

Ignore questions above for now.

[jamieparkinson]

I think that this will probably involve configuring a https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-limit-token-count-tokenfilter.html to be used, quite possibly only at search time https://www.elastic.co/guide/en/elasticsearch/reference/current/search-analyzer.html

[kenoir]

Closing in favour of https://github.com/wellcomecollection/platform/issues/5739