Add search index creation

[Keats]

Find a way to create a search index that can be used by a JS library I guess?

[Keats]

See https://github.com/azerupi/mdBook/issues/51

[Keats]

And see https://github.com/rust-lang-nursery/mdBook/pull/472 for the PR implementing it

[Keats]

How it would work:

• a setting in config.toml like build_search_index defaulting to false

By default it would build search index on every single content in a markdown file. A section could block search index for itself and its pages/subsections by setting something like a in_search_index = false in the front matter. The same setting should exist on page frontmatters to allow more granularity. Search should also ignore draft articles.

The index should be built with elasticlunr-rs as seen in the PR above with titles having a higher weight than paragraph.

Search index should be a question asked in gutenberg init and if yes, setup the config.toml and copy the js library that would have been include_str! in gutenberg somewhere in static.

Probably lots of things will change once it actually gets implemented though

[Keats]

Latest mdbook PR: https://github.com/rust-lang-nursery/mdBook/pull/604

I still think the above is reasonable (bikeshedding on names ok), as long as the documentation to add search is well written. The book theme (https://github.com/Keats/book) could be the test theme for for that as I'm planning to add search to it.

I might start working on it this weekend/next week but if someone wants to do it, please leave a message so work isn't duplicated.

[Keats]

It's also worth thinking how the index would work across several languages: only one index for all I guess?

[mattico]

Hi! You're probably the only person who's thinking about using elasticlunr-rs, so that makes you my most important "customer". Let me know if you have trouble with anything.

In particular, you mentioned languages. There are a few language-specific components to creating a search index. I've only implemented these for English at the moment. Supporting more languages looks straightforward, I've created an issue: https://github.com/mattico/elasticlunr-rs/issues/7. I'll likely work on this at some point if there's interest. Of course, PRs welcome ;)

[Keats]

Hi! You're probably the only person who's thinking about using elasticlunr-rs, so that makes you my most important "customer". Let me know if you have trouble with anything.

Don't sell yourself short, there's also your mdbook PR! I believe the language could be done seamlessly from Gutenberg point of view since we have a default language in the config and the multi-language RFC will include the language code in the filename. I'll comment a bit on that on the issue directly.

Thanks for the offer, I will probably use it ;)

[Keats]

Quick question: it seems like elasticlunr.js is not maintained anymore, is the index generated working with lunr.js as well? I haven't used either of them so have no idea how they compare.

[mattico]

Is it not maintained? That's news to me!

No, the indexes are not compatible. However the author of lunr did make a port of that to Rust which you could check out. I'm on my phone else I'd link it.

[Keats]

Ah no misread a bit that line on the repo: As I'm now focusing on new domain, hope that someone who interested on this project could help to maintain this repository.

Anyway, if it works, it works - it shouldn't be an issue.