Migrate search autocomplete from Algolia to Typesense

[Krinkle]

Background as documented previously:

As of 2021, we're exploring an open-source solution that we can support within the free software ecosystem. In doing so we will increase security and availability (by reducing client-side dependence on third-party domains), and lower our privacy budget.

We first evaluated Meilisearch and experienced some suboptimal aspects. These included: difficult upgrades (not yet committing to forward compatibility or automatic in-place upgrades), opt-out telemetry instead of opt-in, no official Debian packages, non-trivial interactive setup, missing support for querying multiple indexes (e.g. qunitjs.com and api.qunitjs.com), and a not yet clear future in terms of business model (Meilisearch Cloud was not yet in the picture, and the backend is not GPL licensed).

In mid-2022, the experiment transitioned to focus on Typesense instead.

---

Since April 2023 we have an instance of Typesense running in the new infra, provisioned through this repostory (https://github.com/jquery/infrastructure-puppet/commit/558de966f806c18203bfe739778fc895bcb44bf8). I also developed a 2kB minimalistic HTML-first client and user interface for it at https://github.com/jquery/typesense-minibar and integrated it with our Jekyll theme at https://github.com/qunitjs/jekyll-theme-amethyst/. This has been live on https://qunitjs.com/ for the past few months.

Next, we need to migrate the remaining doc sites which are still using the (now stale and deprecated) Algolia DocSearch indexes:

• [x] Set up crawling for qunitjs.com (incl api.qunitjs.com)
• [x] Set up crawling for jquery.com (incl api.jquery.com)
• [x] Set up crawling for jqueryui.com (incl api.jqueryui.com)
• [x] Set up crawling for jquerymobile.com (incl api.jquerymobile.com)
• [x] Integrate typesense-minibar at https://github.com/jquery/jquery-wp-content, test locally via jquery-wp-docker, and theme acordingly for each of the three themes.
• [x] Deploy for https://qunitjs.com and api.qunitjs.com.
• [x] Deploy to staging sites, including https://stage.api.jquery.com, https://stage.jqueryui.com, and https://stage.jquerymobile.com
• [x] Deploy for https://jquerymobile.com and api.jquerymobile.com.
• [x] Deploy for https://jqueryui.com and api.jqueryui.com.
• [x] Deploy for https://jquery.com and api.jquery.com.

[Krinkle]

I've drafted WordPress integration at https://github.com/jquery/jquery-wp-content/tree/draft-typesense.

Visually, the input field is slightly taller, and I also made it a bit wider to make up for the increase spacing so preserve a more balanced feel (I think?).

Search field

Search results

For jQuery Core, the main difference is that we no longer duplicate results from the same page (which is an intentional configuration difference), so that a search like "ajax" won't show 5/5 results of the same page but rather give you 5 different pages to choose from.

For jQuery UI, the results are more or less the same. The improvements I made is that it now follows local brand colors (only requires 2 CSS variables!), and the prevalent link words and QuickNav Examples words are gone from all results by adding .icon-link.toc-link and #quick-nav header h2 to the scraper's selectors_exclude config.

There's also various small match and ranking improvements based on special characters. And of course most significantly, the fact that TypeSense is freely-licensed open source software, the whole minibar widget is only 2kB (compared to ~100 kB), and is served without privacy-leaking third-party requests.

Before	After