search

Pelican-Elegant / elegant

Best theme for Pelican Static Blog Generator
https://elegant.oncrashreboot.com/
MIT License
295 stars 187 forks source link

Replace tipue search with lunr.js and lunr.py #275

Closed codecachet closed 4 years ago

codecachet commented 5 years ago

I am distinctly getting the impression that tipue search is no longer being maintained, or is otherwise less than desirable. I suggest that we replace tipue search with lunr - lunr.py to create the index, and lunr.js to query the index when a search is launched. Both projects seem alive and well, and both Jekyll and Hugo seem to recommend using lunr for search.

Just a thought - like others, I have been very frustrated trying to get search to work for Elegant. One thing about tipue is I am not crazy about the jQuery dependency. I know jQuery is a dependency of Elegant currently, but if we get rid of Bootstrap, we could get rid of jQuery.

talha131 commented 5 years ago

I too want to get rid of jquery. It's on our roadmap.

As for tipue. When elegant was first released, tipue was the only available search solution for static sites. Now we have much better and powerful options like

  1. https://fusejs.io/
  2. https://github.com/fergiemcdowall/search-index
  3. https://lunrjs.com/
  4. https://github.com/nextapps-de/flexsearch

Down the line, we should replace or add support for other search solutions too.

iranzo commented 5 years ago

@codecachet do you want to create a PR for this?

talha131 commented 5 years ago

@iranzo this is a big change. It should be handled by the core team. More importantly, we need to compare different solutions first.

talha131 commented 5 years ago

An issue noticed with tipue is that it treats plural differently from singular

Search for "variable" and then "variables" in the docs site. Result is different.

We can use stem words option to fix it but then how many words are we going to add in the list. Tipue should have this feature by default.

iranzo commented 4 years ago

I'm looking at lunr.js for other aspects, seems that the data it uses for the search can be contained/generated on publish time without plugin, just a jinja loop (at least on jekyll). I'll provide some information once I finish my investigation.

talha131 commented 4 years ago

@iranzo I appreciate that you have taken up this task. Before you go ahead and look deeply into lunr.js, can you please step back and decide which library we should use.

In terms of Github stars, Fuse is way ahead of lunr.js, which is followed by flex search.

Github stars are not a reliable metric but they do give a pretty good idea about the community interest and support, which we are going to need.

Then there is performance factor. Flex claims that it is the most performant.

Finally, we have to consider the ease of use for the end user.

Can you please investigate these three libraries and let me know how they fare on the criteria I mentioned?

iranzo commented 4 years ago

Hi, No worries, just wanted to mention that I was testing it as a website I'm using has it integrated (but it was pretty old version and using jekyll) about features. Lunr is not that bad, but my concern here is that it, as with tipue, pregenerates a javascript array with all website content... that means that for a big website, it will have to preload all information in the html generated and then perform the search.

I don't mind looking at flex directly to see how it performs, but probably doing perf test will require lot of articles and I was thinking that some one here in the team had a busy one that could be used for testing if the website source is available somewhere.

talha131 commented 4 years ago

This is the result of flex search benchmark that I got.

Flex is much more performant than other libraries. How well it is supported and how actively it is under development is to be investigated.

talha131 commented 4 years ago

that means that for a big website, it will have to preload all information in the html generated and then perform the search.

You hit the nail right on the head.

If we can avoid it using these new libraries then it would become a high priority feature to implement and ship.

talha131 commented 4 years ago

Flex search is fast but development has seem to have lost momentum.

  1. https://github.com/nextapps-de/flexsearch/graphs/commit-activity
  2. https://github.com/nextapps-de/flexsearch/issues/150

Also, the documentation is not friendly. We will have a hard time making a head and tail of it.

talha131 commented 4 years ago

that means that for a big website, it will have to preload all information in the html generated and then perform the search.

I just tested lunrjs demo. Even if we have to ship a JSON object with all the contents of the site, just like TipueSearch, lunarjs has better search results.

Try searching for "assignment" in the demo here It shows "assigned" in the results too, which is desirable.

Tipue cannot handle plural and singular words correctly.

I think lunrjs is a better option.

talha131 commented 4 years ago

Questions and work items for whoever pick up this task.

iranzo commented 4 years ago

From what I've seen about lunr.js: The result is generated by jekyll when the page for 'search' is created, it contains array for all the text in the website.

Search page replaces text in the 'search results' area with the output of the script

Right now the tipue_search plugin does the generation of the search results as lunr just uses a backend for the index that needs to be populated it could also probably be done with current tipue_search plugin output or even try to avoid that plugin to get one 'easier' by no longer requiring ap plugin (if possible), as I've tested this with jekyll and not with pelican

talha131 commented 4 years ago

or even try to avoid that plugin to get one 'easier' by no longer requiring ap plugin (if possible)

That's not a good idea. A plugin improves search performance. is what lunrjs docs recommends,

https://lunrjs.com/guides/index_prebuilding.html

talha131 commented 4 years ago

How to generate search result from lunr.js

  1. https://codewithhugo.com/hugo-lunrjs-search-index/
  2. https://matthewdaly.co.uk/blog/2015/04/18/how-i-added-search-to-my-site-with-lunr-dot-js/
  3. https://learn.cloudcannon.com/jekyll/jekyll-search-using-lunr-js/
talha131 commented 4 years ago

The json generated by Tipue Search plugin can be used by lunrjs without any changes.

All we need is to generate the search results page.

If anyone is willing to step up to write the code for search results page, then please let us know. I will work with you to save your time and effort, if needed.

iranzo commented 4 years ago

@talha131 review #573

Note, the code just 'works', sytling should be used on the results functions and probably remove the tipue search javascript from website (and maybe put the lunr one)

I count on you for the 'styling' , in the current form for the PR, it works as a drop-in replacement of tipue with same requirements (for plugin, etc)

talha131 commented 4 years ago

Love it. Will do the rest.

iranzo commented 4 years ago

image

talha131 commented 4 years ago

:tada: This issue has been resolved in version 5.2.0 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket:

dclong commented 3 years ago

Does elegant still uses the tipue_search pelican plugin and only replaces the underlying tipue with lunr.js? Is configuration still the same?

talha131 commented 3 years ago

yeah. underlying configuration is same