Highlighter mergeContiguousFragments shouldn't merge 0-score fragments [LUCENE-6471]

[asfimport]

Highlighter.mergeContiguousFragments merges adjacent fragments it is given. But it is given a list of fragments that do not necessarily have embedded highlights (e.g. have a score of 0), and so it could grow a fragment needlessly. I never figured out why this old highlighter keeps around such fragments instead of eagerly tossing them when the fragment completes, which is what I think it should do. That would address this problem and might make things faster. I'm not sure if any highlighter user wants the non-scoring fragments though.

---

Migrated from LUCENE-6471 by David Smiley (@dsmiley), 1 vote, updated Jul 11 2017

[asfimport]

Jim Richards (migrated from JIRA)

I ended up writing this to get around this issue

    public final String getBetterFragments(TokenStream tokenStream, String text, int maxNumFragments, String separator) throws IOException, InvalidTokenOffsetsException {

        maxNumFragments = Math.max(1, maxNumFragments); // sanity check
        TextFragment[] frag = getBestTextFragments(tokenStream, text, false, maxNumFragments);

        StringBuilder result = new StringBuilder();
        for (int i = 0; i < frag.length; i++) {
            if (frag[i] != null && frag[i].getScore() > 0) {
                if (i > 0) {
                    result.append(separator);
                }
                result.append(frag[i].toString());
            }
        }
        return result.toString();

    }

[asfimport]

David Smiley (@dsmiley) (migrated from JIRA)

Jim Richards Have you considered the UnifiedHighlighter?

[asfimport]

Jim Richards (migrated from JIRA)

@dsmiley,

I'm using hibernate-search-orm, which is using 5.5.4 of the Lucene libraries and UnifiedHighlighter is from 6.x. I could probably do some magic in pom.xml to exclude 5.5.4 but my basic test had too many incompatibilities.

Jim.