Highlighted matched terms

[cbou]

It would be nice if Lunr.js could highlight matched terms.

[olivernn]

Do you mean being able to wrap matched terms in some markup which can be later styled, e.g.?

This is a <span class="lunr-match">matched</span> term

I think this would be cool, although currently I'm not sure it's possible. There are a couple of things that lunr needs to be doing before this can be achieved, also I think the functionality for actually marking up matched terms is probably better handled outside of lunr, in some kind of plugin.

What lunr could do is return the positions of the matches in a particular document. Currently lunr discards the position of words in the documents that it indexes. Making use of the positions of terms is something that I've been thinking about a little bit, it would allow for more powerful queries, e.g. documents with "hello" and "world" next to each other.

I think this might be a little way off yet, but its definitely something that I'm interested in adding. If you have any thoughts on implementations then I'd be glad to hear them too.

[michaelmior]

Being able to get the position of the result in a document would be great. lunr is closest thing I've found to what I need for a new project, but I really need the ability to get the position of the matched word in the document. Any pointers on where to start would be great. I assume this could be done fairly simply by changing the index format.

Anyway, once positions are tracked, I'd imagine an interface for highlighting would be fairly straightforward.

[sergey-tihon]

Agreed, highlight should be a killer feature.

[andrewwakeling]

+1. It's vital to provide context for matches especially when indexing multiple fields.

[Stephenitis]

+1. Agreed trying to make the results user friendly is top priority.

[ysadka]

+1 would be great to have this!

[EricLongpre]

+1. Having the concept of matched tokens is definitely required. This is major for expanding to awesome usability. Without it, we'd have to try and reverse engineer the search logic to attempt to show the user what exactly we found to merit the results.

[JaimeObregon]

:+1:

[gouldingken]

Does the matching and storing of tokens with position have to happen on the indexing side? When populating search results, you typically have the full text. The only complex part is figuring out how stemmed or other loosely matched terms are being handled. For direct matches you can basically just do a split and join on the search term, truncate it appropriately and wrap it in some styling.

Would it be possible to have a method taking the full text and search term that would return the positions? I imagine it would basically use the tokenizers and stemming algorithms to find the location for the given string. That seems simpler (and less memory intensive) than storing every token and its original position as part of the indexing process.

[julkue]

I would like to realize this with a highlighting component. However, first of we need to make sure that highlighted words and matches by lunr are exactly the same for a good usability concept. Therefore I created https://github.com/olivernn/lunr.js/issues/200.

[olivernn]

I have published an alpha release of the next version of lunr with support for highlighting matched search terms.

In addition I've put together a basic demo showing, amongst other things, highlighting of matched terms.

The intention of this demo and alpha release is to get some feedback on the interface that lunr provides to aid highlighting search terms. Any and all feedback is welcome.

[MykolaGolubyev]

Wonderful demo. Will try to make it work for my case. In my case I am extracting text from more complex structures to feed the indexer. Later I don't have the extracted text, but only the original data structure. So in order to highlight I have to repeat the "indexing" process. The problem is I am doing indexing on a server, but search is purely on a client...

[MykolaGolubyev]

Doing some progress with it. Would you prefer to get feedback on resulting structure here or do you have other channels?

[olivernn]

@MykolaGolubyev I think open a new issue, and just reference this issue, so there is still a link. Thanks for taking the test things out, all feedback is super useful.

[julkue]

@olivernn I would investigate this too if you could let me know if you'll make the highlighting component (wrapper.js) a full-working solution? Then I don't think there would be a need for mark.js.

[MykolaGolubyev]

Mark is still a solution for my case. Text I display and text I index are not strictly the same.

[julkue]

@MykolaGolubyev Thanks for your reply. Could you please elaborate this a bit? I can't fully imagine the use case.

[MykolaGolubyev]

I have a data structure that represents a rich text content. i have many blocks of those on different pages. When I create an index I extract certain text from them and associate with a block. Then I extract different kind of text and give it a higher weight and again associate with the block. When I search I find an associated block. Navigate there and highlight certain words in an already rich text. To do that I convert positions to unique words and call mark.js. So I have to re-extract text from that block first. At the moment I share code with the server side (Nashorn) to achieve that.

[olivernn]

@julmot If there are already libraries out there that do the job and handle browser inconsistencies correctly (mark.js) then I'd be much more inclined to look at ways of integrating with them instead of creating (and maintaining) another library.

I'd be interested in working with you on a way of providing some Lunr support in mark.js, what is the best way to collaborate on that, an issue here or on mark.js?

[julkue]

@olivernn Thanks for your reply. Sure, we can discuss about this. Please open a new issue, then we see if a plugin is helpful (separate repository).

@MykolaGolubyev If I understand you correctly you're building your search index by extracting text from pages but don't use this search index to display the results. That's why mark.js is helpful here. Am I right?

[MykolaGolubyev]

Correct. With one amend: I am not extracting text from pages. I am using a data source to build a page and to prepare text for indexing.

[julkue]

Hey @olivernn,

As v2 is now released, I just would like to ask you about the status of highlighting matches for this release. Did you decide to go with mark.js or do you recommend a different way?

Thanks for your reply.

[nylen]

I didn't see this mentioned anywhere that was easy to find, so as of the current version (2.3.8) lunr makes it easy to include the position as part of the index. Reference: https://github.com/olivernn/moonwalkers/blob/6d5a6e976921490033681617e92ea42e3a80eed0/build-index#L23-L29

With my dataset this increases the index size about 33%, from 2.1 MB to 2.8 MB.

[lrq3000]

Why is this issue still open if it's now implemented? Could we get a formal doc about this feature and how to implement it please? Awesome work and feature BTW, lunr is awesome! :D

[FaruqueBraimo]

Hi @olivernn , how we can do this, with current version?

[zhengchun]

Hi everyone, I found a solution how to highlight multiple search keywords that using document.createTreeWalker() and document.createRange(), this method without use position of terms. You can visit this demo site FastBootstrap to search avatar, border or anthoer keywords.

The code snippet below from Fastbootstrap's search.js file:

function highlightTokens(element, tokens) {
    const walker = document.createTreeWalker(element, NodeFilter.SHOW_TEXT, nodeFilter, false);
    let node = null;

    while ((node = walker.nextNode())) {
      const text = node.textContent.toLowerCase();
      let found = false;
      for (var i = 0; i < tokens.length && !found; i++) {
        const token = tokens[i].toString();
        const startIndex = text.indexOf(token);
        if (startIndex == -1) {
          continue;
        }
        let range = document.createRange();
        range.setStart(node, startIndex);
        range.setEnd(node, startIndex + token.length);
        let mark = document.createElement('mark');
        range.surroundContents(mark);
        found = true;
      }
      walker.nextNode();
    }
}

A caller method:

let $title = document.createElement('div');
$title.innerText = "Avatar group";
highlightTokens($title, ["avat"]);
console.log($title.outerHTML);

output:

<div><mark>avat</mark>tar group</div>