search

JuliaDocs / Documenter.jl

A documentation generator for Julia.
https://documenter.juliadocs.org
MIT License
814 stars 482 forks source link

Different search backends #615

Closed haampie closed 12 months ago

haampie commented 6 years ago

Since search performance is questionable I tried indexing the official Julia docs in Algolia, which works pretty well. See http://julia-search-docs.surge.sh/ for an example. This was created by scraping the web page, not by generating the index with Documenter.jl.

Algolia offers this service for open source projects: https://community.algolia.com/docsearch/

I was thinking it is useful to allow differ search backends when generating docs. Thoughts?

haampie commented 6 years ago

My experience with Algolia so far is pretty good. You can search on macros like @time and @., on symbols like :LR, it has typo tolerance, responds in just 20ms etc.

haampie commented 6 years ago

Also possible is pagination and facetting:

https://haampie.github.io/julia-docs-search-ui/

For this example I've generated the index via Documenter.jl. I'm still looking for a way to get sensible excerpts from the content; right now the markup is removed.

mortenpi commented 6 years ago

Improvements to the search are certainly welcome. I wasn't aware of Algolia, but it looks pretty awesome. I am surprised how well it is able to scrape and make sense of the HTML already.

We could default to the current lunr.js-based for packages and offline docs, but allow integration with Algolia through an option (we need to pass the API keys anyway). As far as I can tell, it shouldn't be too difficult to integrate this into the current search page.

Would you be willing to take a stab at putting a PR together for this? The examples you have seem like pretty solid proofs of concept. I'd be more than happy to review and help with the Documenter side.

haampie commented 6 years ago

Would definitely like to contribute!

As was remarked on discourse there are two parts to this: an improved search functionality and an improved user interface as well.

I'm using Bootstrap for the latter, which makes life really easy. Maybe that would be useful for the docs as a whole as well? It would also allow theming in other packages.

mortenpi commented 6 years ago

I'm using Bootstrap for the latter, which makes life really easy. Maybe that would be useful for the docs as a whole as well? It would also allow theming in other packages.

Bootstrap basically provides a bunch of UI components, right? I would prefer if the docs wouldn't depend on JavaScript (the basic functionality should fall back nicely).

The search, of course, needs JS anyway. But for the style to be consistent with the rest of the docs I am wondering whether it would just be easier to do the the search UI the same way?

The philosophy with the HTML/CSS/JS design has been to keep it as simple and standard as reasonably possible. This includes keeping the HTML structure of the page simple, e.g. by avoiding superfluous <div>s etc.

haampie commented 6 years ago

Totally agree with keeping the docs minimalistic.

Bootstrap does not require Javascript, and via SASS you can generate only those parts of the CSS you want to use, customizable with variables. When the HTML is 'idiomatic' Bootstrap, then #579 is easy. In particular when the assets and tooling (sass -> css, minifier etc) for it are moved to a separate repository. Probably issues like #618 will not show up neither.

My motivation to have Bootstrap (the HTML en CSS that is) all over the place is that it's easier to add a new feature like search (without any knowledge of design / CSS)

mortenpi commented 6 years ago

I took a closer look at Bootstrap and I am intrigued now. It would indeed be great to offload as much of the design work to a library like that. Basically we'd just have to annotate a bunch of the HTML tags we have with the various Bootstrap classes.. and translate the current theme to Bootstrap, right?

For SASS, are you proposing building the theme in a separate library and distributing the compiled CSS with Documenter's package? FWIW, since SASS is just a Ruby gem, it might be fine for Documenter to depend on it as well. Installing it on Travis shouldn't be a problem.

haampie commented 6 years ago

Basically we'd just have to annotate a bunch of the HTML tags we have with the various Bootstrap classes.. and translate the current theme to Bootstrap, right?

Exactly

Maybe it's best to move this discussion to a new issue

odow commented 12 months ago

Closing as stale. We have a new search backend now.