Need help update the Fuse.js dependency to 5.x or later

[kaushalmodi]

Currently, this Hugo module is fetching Fuse.js v3.2.1. Attempting to upgrade to 5.0.4-beta or later break the search.

The breakage is explained in https://github.com/krisk/Fuse/releases/tag/v5.0.4-beta. I can use some help from people understanding JavaScript to help fix https://github.com/kaushalmodi/hugo-search-fuse-js/blob/main/assets/js/search.js so that the latest Fuse.js works.

Steps to reproduce the issue

Update Fuse.js to the latest version locally

git clone https://github.com/kaushalmodi/hugo-search-fuse-js
cd hugo-search-fuse-js
hugo mod get -u 

Run test search

cd exampleSite
./srv.sh

Open http://localhost:2323/search/?q=heading in your browser and you will see that that search doesn't return anything.

Reverting back to the working version

From another terminal, cd to the repo root and run hugo mod get github.com/krisk/Fuse@ec5d1a7079a03784e7a3e6778b428cad8774d881 to revert Fuse.js to v3.2.1 and you will see that http://localhost:2323/search/?q=heading will now be displaying some results.

[kaushalmodi]

@Robin-Wils Just checking if fixing this falls in your domain. If so, can you please take a look?

[kaushalmodi]

@Robin-Wils https://github.com/kaushalmodi/hugo-search-fuse-js/pull/16#issuecomment-1027263087

Ah, I see also the high Fuse.js problem. https://www.robinwils.com/js/libs/fuse.min.e3ffe275857f00d215368b38a182ac1950c7249fe2e7b019daebada5dfe79afd.js

Will check tomorrow evening, also did a rollback. It might be possible to just take what we need instead of the whole library.

Thanks! Let's continue that discussion in this issue.

[Robin-Wils]

https://github.com/krisk/Fuse/issues/428 Should be not too hard to fix, mapping the objects. Tokenized might have been removed though, so that option should be removed too.

Although there might be other libraries too, since those comments suggest that it isn't the first time it has breaking changes.

[Robin-Wils]

Fuse is quiet big, there are many alternatives. Listjs, seemed alright, but not sure how to limit the content there, aside from that it can do pretty much everything fuse used to do. Fuzzysort is the next one I will look at. https://stackify.dev/javascript-fuzzy-search-that-makes-sense

[kaushalmodi]

Don't worry about it. The locked version of Fuse.js is works.. so it's alright. I'll ask on the Fuse.js repo for help.

[Robin-Wils]

I managed to get horsey working, which is a lot lighter than Fuse.js. Although it doesn't seem maintained anymore. I might mail the developer later.

Fuse.js can be used, but I think I might fork this repo, depending of I get a mail back. I prefer better performance. Fuse gives better matches, but is much heavier on the resources. Some files of both repos could become modules, since there will still be files which are needed for both.

There are fixes, in the form of pull-requests, in the horsey repo, for the problems I still get with highlighting (case-sensitive).

Example (runs without building, just as html. It does not fetch data for testing purposes):

<div class="search"> 
  <input autofocus=""> 
  <button class="filter-item" style="font-weight:700;border:0" type="submit">Search</button> 
  <hr> 
</div>
<script src="https://bevacqua.github.io/horsey/dist/horsey.js"></script> 
<style>
  .sey-char-highlight{background-color:yellow}
  .sey-list {list-style-type:none;padding:0;margin:0}
</style>

<script>
const loadFile = [{"categories":null,"contents":" Contact Ask me anything You can ask me anything: Ask me anything. Email Email: mrwils@tutanota.com - the best way to reach me. Twitter and Facebook - social media things Note: I barely check these. I am not a huge social media fan. Twitter: @wils_robin Facebook: Robin Wils Letterboxd - film reviews Letterboxd: RobinWils Curriculum Vitae Curriculum Vitae  No longer active social media  @RobinWils@gleasonator.com Gleasonator - Twitter and Facebook alternative. @RobinWils@letsalllovela.in Let's All Love Lain - Twitter and Facebook alternative. @RobinWils LBRY - video platform. Robin Wils YouTube - video platform. Robin Wils Twitch - live streaming platform. @wils_robin Instagram - picture sharing platform. @rmw@pixfed.com Pixelfed - instagram alternative.   ","permalink":"https://www.robinwils.com/contact/","tags":["sci-fi", "test"],"title":"Contact"}]

horsey(document.querySelector('input'), {
  source: [{ list:  loadFile}],
  getText: 'title',
  getValue: 'content',
  highlighter: true,
  highlightCompleteWords: true,
  limit: 15,
  //noMatches:'Please enter a word or phrase above',
  renderItem: function (li, suggestion) {
    let tags = ""
    if (suggestion.tags != null) {
      tags = '<p><strong>Tags:</strong> ' + suggestion.tags.join(", ") +'<p>'
    }

    let categories = ""
    if (suggestion.categories != null) {
      categories = '<p><strong>Categories:</strong> ' + suggestion.categories.join(", ") +'<p>'
    }

    li.innerHTML = '<div class="search_summary">' +
    '<h2 class="no-text-decoration">' + suggestion.title +' <a href="' + suggestion.permalink + '"></a> </h2>' +
    '<p><i>' + getSnippet(suggestion.contents) + '</i></p>'+
    tags +
    categories
  }
});

function getSnippet(contents) {
    let snippet = "";
    let searchQuery = document.querySelector('input').value;
    // How many characters to include on either side of match keyword
    let summaryInclude=60;

    if (snippet.length < 1) {
      var getSentenceByWordRegex = new RegExp( 
        `[^.?!]*(?<=[.?\\s!])${searchQuery}(?=[\\s.?!])[^.?!]*[.?!]`,
        'i'
      );
      var maxTextLength = summaryInclude*2
      // Index of the matched search term
      var indexOfMatch = contents.toLowerCase().indexOf(searchQuery.toLowerCase());
      // Index of the first word of the sentence with the search term in it
      var indexOfSentence = contents.indexOf(getSentenceByWordRegex.exec(contents));

      var start 
      var cutStart = false
      // Is the match in the result?
      if (indexOfSentence+maxTextLength < indexOfMatch) {
        // Make sure that the match is in the result
        start = indexOfMatch
        // This bool is used to replace the first part with '...'
        cutStart = true 
      } else {
        // Match is in view, even if we show the whole sentence
        start = indexOfSentence
      }

      // Change end length to the text length if it is longer than 
      // the text length to prevent problems
      var end = start + maxTextLength 
      if (end > contents.length) {
        end = contents.length
      }

      if (cutStart) {
        // Replace first three characters with '...'
        end -= 3;
        snippet += "…" + contents.substring(start, end).trim();
      }
      else {
        snippet += contents.substring(start, end).trim();
      }     
    }
    snippet += "…";
    return snippet;
  }
</script>```

[kaushalmodi]

I might fork this repo

Please do that! Anyways this repo is named "xx-fuse-js". Can you create a hugo-search-horsey or similar Hugo Module? I would like to try it out.

[kaushalmodi]

@Robin-Wils A related search library that has come back on my radar is Lunr.js.

• Here's a Hugo theme repo by https://tabletop.social/@takeonrules : https://codeberg.org/takeonrules/takeonrules-hugo-theme
• The search on his site is pretty spiffy: https://takeonrules.com/search/

May be you or someone can extract that to hugo-search-lunr-js Hugo Module?

[Robin-Wils]

Yeah, Lunr is popular, lighter than fuse too (in filesize). Not as light as the one in the html above on resources though, well not likely. The one in the html code above is made to not do regex or whatever which makes it light on resources. Horsey use this in the background: https://github.com/bevacqua/fuzzysearch Disadvantage: results are not as accurate as some other libs, it doesn't add a 'score' to something

I don't need too much in a search, so above would be fine in my case. Lunr could be looked at too. Probably not too hard to implement, since it is popular, so likely user-friendly. It would be cool to have multiple choices. There is also some lib which does something like sublime search, which some people really like.

[Robin-Wils]

Sorry that I haven't done it yet. Haven't found the time... and honestly there are some other things I want to spend some time on. Will try to look at it when I got some time in between something.

The basic code for horsey is posted above, haven't looked at lunr yet.

[Robin-Wils]

Haven't got a mail back. Might try to contact the maintainer on Twitter or something. Fuse.js works though. So search actually functions fine for me. Functionality is more important than how light it is.

Oh, I also noticed that I can find my search.html through the search, maybe that should be excluded from the json.

[Robin-Wils]

https://gohugo.io/tools/search/ There are some existing options it seems. Fuse seems somewhat old according to some comments, but it works.

https://github.com/riesinger/hugo-golunr This seems interesting too.