Search: cross-site search federation

josegonzalez commented 1 year ago

Context

I have multiple mkdocs-backed doc sections that compose the Dokku project:

main docs
tutorials subsite
etc.

All of these have their own release cycle and thus are in different repositories, but more or less roll up to the same "dokku/docs" repo for their generated contents. This allows me to have quicker CI runs for each while still keeping on the same domain.

One thing that isn't great is that search results are scoped to just the section a user happens to be on. It is all effectively one "site" but because of how the sites are generated, we don't show the related documentation from each sub-site.

Description

What I would like to do is be able to configure mkdocs such that it can either:

Point to several search indexes
I can manually merge the indexes and specify a custom "site_dir" for each search result

If several indexes are merged, a user should be able to specify some config:

an index title
an optional favicon to show next to the index title

This would then appear in search results in a more unified way, with tabs for each index.

Use Cases

Maybe a user could configure other search indexes to load like so:

plugins
  - search:
      additional_indexes:
        - title: Documentation
           site_index: https://example.com/docs/
           index_url: https://example.com/docs/search/search_index.json
        - title: Tutorials
           site_index: https://example.com/tutorials/
           index_url: https://example.com/tutorials/search/search_index.json

When building search results, the results from these additional indexes would have their site_index injected and then that would be used when templating out results to provide a link to the docs. Search would maybe have to be refactored a bit to run against several indexes at once (not sure how that would work, was a bit hard to grok how search is launched from a quick browse of the source).

From a developer's perspective, the yaml config is all they would need to completely spider a site (or multiple sites) and have their results show up. For users, this unified search would provide a much better experience when browsing multiple distinct doc results.

Visuals

Below is something I'm hoping to approximate, from the Nomad documentation.

Before submitting

[X] I have read and followed the change request guidelines.
[X] I have verified that my idea is a change request and not a bug report.
[X] I have ensured that, to the best knowledge, my idea will benefit the entire community.
[X] I have included relevant links to the documentation, related issues and discussions to underline the need for my idea.

squidfunk commented 1 year ago

Thanks for suggesting! We'll be considering this feature request for the next iteration of search that we'll start working on later this year. This will make search more flexible and adaptable to various use cases like yours.

damienrj commented 1 year ago

Is this still up for consideration?

squidfunk commented 1 year ago

Yes, this will be part of the new search implementation that we're currently working on. We just need to first finish the work on the 9.2 release, then we'll continue working on the new search. It will offer a federated mode where you can define multiple sites to pull search indexes from that will be merged appropriately.

damienrj commented 1 year ago

Great, looking forward to seeing it in our insider's version 👏

squidfunk commented 1 year ago

Please see the announcement in #6307.

strausmann commented 1 year ago

It would be great if you could use relative urls. as the base domain will differ depending on our environment.

AbhishekGS2021 commented 1 month ago

Context

I have multiple mkdocs-backed doc sections that compose the Dokku project:

main docs

tutorials subsite

etc.

All of these have their own release cycle and thus are in different repositories, but more or less roll up to the same "dokku/docs" repo for their generated contents. This allows me to have quicker CI runs for each while still keeping on the same domain.

One thing that isn't great is that search results are scoped to just the section a user happens to be on. It is all effectively one "site" but because of how the sites are generated, we don't show the related documentation from each sub-site.

Description

What I would like to do is be able to configure mkdocs such that it can either:

Point to several search indexes

I can manually merge the indexes and specify a custom "site_dir" for each search result

If several indexes are merged, a user should be able to specify some config:

an index title

an optional favicon to show next to the index title

This would then appear in search results in a more unified way, with tabs for each index.

Related links

I looked to see if this was in the discussion board and docs, and that doesn't seem to be the case.

Use Cases

Maybe a user could configure other search indexes to load like so:
plugins
  - search:
      additional_indexes:
        - title: Documentation
           site_index: https://example.com/docs/
           index_url: https://example.com/docs/search/search_index.json
        - title: Tutorials
           site_index: https://example.com/tutorials/
           index_url: https://example.com/tutorials/search/search_index.json
When building search results, the results from these additional indexes would have their site_index injected and then that would be used when templating out results to provide a link to the docs. Search would maybe have to be refactored a bit to run against several indexes at once (not sure how that would work, was a bit hard to grok how search is launched from a quick browse of the source).

From a developer's perspective, the yaml config is all they would need to completely spider a site (or multiple sites) and have their results show up. For users, this unified search would provide a much better experience when browsing multiple distinct doc results.

AbhishekGS2021 commented 1 month ago

@squidfunk When can we expect federated search functionality will be released ? . I see everyone are eagerly waiting. As this function makes searching simpler.

squidfunk commented 1 month ago

Please read our latest blog post, where we explain why things are taking longer than expected. Also, please refer to #6307 for updates, which is where we will announce then availability.

squidfunk / mkdocs-material