alphagov / search-api

Search API for GOV.UK
https://docs.publishing.service.gov.uk/apps/search-api.html
MIT License
31 stars 9 forks source link

Disable Learning to Rank's involvement in queries #2905

Closed sihugh closed 3 months ago

sihugh commented 3 months ago

Learning to Rank only changes the ordering of results when the query includes keywords and is ordered by relevance.

It can only affect how results rank within a single page. The maximum effect it could have is to reverse the order of a page of results. The results all still appear on the same page.

Site search searches with keywords (the majority of searches) are now served by Vertex AI search. The cost and complexity of running a Learning to Rank training and serving pipeline for the other use cases no longer seems commensurate with the impact.

In addition to this, the training of our model has not been supervised effectively for a while, leading to a drop in the quality of the model.

I therefore propose disabling the use of Learning to Rank in Search-API (this change) which allows the removal of the code to invoke the model, the pipeline that builds and deploys the model and the Terraform to support the deployment.

sihugh commented 3 months ago

A side effect of this is a fall in response times for keyword queries (in particular) because we:

KludgeKML commented 3 months ago

This also starts clearing up the tech debt where marking the ab_test parameter is the only way to turn off LTR on an individual finder (the finder that replaced licence-finder had to do this to make the results usable)