BlendedInfixSuggester to handle multi term matches better [LUCENE-8347]

mikemccand commented 6 years ago

Currently the blendedInfix suggester considers just the first match position when scoring a suggestion. From the lucene-dev mailing list : " If I write more than one term in the query, let's say "Mini Bar Fridge" I would expect in the results something like (note that allTermsRequired=true and the schema weight field always returns 1000)

Mini Bar Fridge something
Mini Bar Fridge something else
Mini Bar something Fridge
Mini Bar something else Fridge
Mini something Bar Fridge ... Instead I see this: - Mini Bar something Fridge - Mini Bar something else Fridge - Mini Bar Fridge something - Mini Bar Fridge something else - Mini something Bar Fridge ... After having a look at the suggester code (BlendedInfixSuggester.createCoefficient), I see that the component takes in account only one position, which is the lowest position (among the three matching terms) within the term vector ("mini" in the example above) so all the suggestions above have the same weight " Scope of this Jira issue is to improve the BlendedInfix to better manage those scenarios.

Legacy Jira details

LUCENE-8347 by Alessandro Benedetti (@alessandrobenedetti) on Jun 04 2018, updated Jan 23 2020 Attachments: LUCENE-8347.patch (versions: 2) Linked issues:

LUCENE-8343

mikemccand commented 6 years ago

Patch is attached. It is recommended to go first with LUCENE-8343

[Legacy Jira: Alessandro Benedetti (@alessandrobenedetti) on Jun 04 2018]

mikemccand commented 6 years ago

It is recommended to merge this one first

[Legacy Jira: Alessandro Benedetti (@alessandrobenedetti) on Jun 04 2018]

mikemccand commented 6 years ago

❌ -1 overall

Vote	Subsystem	Runtime	Comment
Prechecks
+1	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
master Compile Tests
+1	compile	0m 48s	master passed
Patch Compile Tests
-1	compile	0m 39s	suggest in the patch failed.
-1	javac	0m 39s	suggest in the patch failed.
-1	Release audit (RAT)	0m 39s	suggest in the patch failed.
-1	Check forbidden APIs	0m 39s	suggest in the patch failed.
-1	Validate source patterns	0m 39s	suggest in the patch failed.
Other Tests
-1	unit	0m 9s	suggest in the patch failed.
	1m 46s

Subsystem	Report/Notes
JIRA Issue	LUCENE-8347
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12926417/LUCENE-8347.patch
Optional Tests	compile javac unit ratsources checkforbiddenapis validatesourcepatterns
uname	Linux lucene1-us-west 3.13.0-88-generic #135-Ubuntu SMP Wed Jun 8 21:10:42 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Build tool	ant
Personality	/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
git revision	master / c587598
ant	version: Apache Ant(TM) version 1.9.3 compiled on April 8 2014
Default Java	1.8.0_172
compile	https://builds.apache.org/job/PreCommit-LUCENE-Build/27/artifact/out/patch-compile-lucene_suggest.txt
javac	https://builds.apache.org/job/PreCommit-LUCENE-Build/27/artifact/out/patch-compile-lucene_suggest.txt
Release audit (RAT)	https://builds.apache.org/job/PreCommit-LUCENE-Build/27/artifact/out/patch-compile-lucene_suggest.txt
Check forbidden APIs	https://builds.apache.org/job/PreCommit-LUCENE-Build/27/artifact/out/patch-compile-lucene_suggest.txt
Validate source patterns	https://builds.apache.org/job/PreCommit-LUCENE-Build/27/artifact/out/patch-compile-lucene_suggest.txt
unit	https://builds.apache.org/job/PreCommit-LUCENE-Build/27/artifact/out/patch-unit-lucene_suggest.txt
Test Results	https://builds.apache.org/job/PreCommit-LUCENE-Build/27/testReport/
modules	C: lucene/suggest U: lucene/suggest
Console output	https://builds.apache.org/job/PreCommit-LUCENE-Build/27/console
Powered by	Apache Yetus 0.7.0 http://yetus.apache.org

This message was automatically generated.

[Legacy Jira: Lucene/Solr QA on Jun 05 2018]

mikemccand commented 6 years ago

New patch attached to fix the minor precommit comments issues.

[Legacy Jira: Alessandro Benedetti (@alessandrobenedetti) on Jun 06 2018]

mikemccand commented 6 years ago

✔ +1 overall

Vote	Subsystem	Runtime	Comment
Prechecks
+1	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
master Compile Tests
+1	compile	0m 48s	master passed
Patch Compile Tests
+1	compile	0m 47s	the patch passed
+1	javac	0m 47s	the patch passed
+1	Release audit (RAT)	0m 47s	the patch passed
+1	Check forbidden APIs	0m 47s	the patch passed
+1	Validate source patterns	0m 47s	the patch passed
Other Tests
+1	unit	1m 37s	suggest in the patch passed.
	6m 20s

Subsystem	Report/Notes
JIRA Issue	LUCENE-8347
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12926717/LUCENE-8347.patch
Optional Tests	compile javac unit ratsources checkforbiddenapis validatesourcepatterns
uname	Linux lucene1-us-west 3.13.0-88-generic #135-Ubuntu SMP Wed Jun 8 21:10:42 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Build tool	ant
Personality	/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
git revision	master / a82c9cf
ant	version: Apache Ant(TM) version 1.9.3 compiled on April 8 2014
Default Java	1.8.0_172
Test Results	https://builds.apache.org/job/PreCommit-LUCENE-Build/28/testReport/
modules	C: lucene/suggest U: lucene/suggest
Console output	https://builds.apache.org/job/PreCommit-LUCENE-Build/28/console
Powered by	Apache Yetus 0.7.0 http://yetus.apache.org

This message was automatically generated.

[Legacy Jira: Lucene/Solr QA on Jun 07 2018]

mikemccand commented 6 years ago

Added some additional edge cases tests + bugfixes :

- assertThat(responses.get(8).key, is("Bar Fridge Mini")); Management of shuffled position in the suggestion but all terms match

term query repetition properly managed

[Legacy Jira: Alessandro Benedetti (@alessandrobenedetti) on Jun 07 2018]

mikemccand commented 6 years ago

Hello,

I am new to this repository. I would like to apply the patch. would you please tell me the right branch to pull to eclipse. Because, there are so many branches and I am confused in order to build.

Any help regarding solr blendedsuggester would be approciated.

Best Regards !

[Legacy Jira: Akshay Patil on Sep 07 2018]

mikemccand commented 6 years ago

✔ +1 overall

Vote	Subsystem	Runtime	Comment
Prechecks
+1	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
master Compile Tests
+1	compile	0m 19s	master passed
Patch Compile Tests
+1	compile	0m 20s	the patch passed
+1	javac	0m 20s	the patch passed
+1	Release audit (RAT)	0m 20s	the patch passed
+1	Check forbidden APIs	0m 20s	the patch passed
+1	Validate source patterns	0m 20s	the patch passed
Other Tests
+1	unit	0m 43s	suggest in the patch passed.
	2m 46s

Subsystem	Report/Notes
JIRA Issue	LUCENE-8347
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12926717/LUCENE-8347.patch
Optional Tests	compile javac unit ratsources checkforbiddenapis validatesourcepatterns
uname	Linux lucene1-us-west 4.4.0-130-generic #156~14.04.1-Ubuntu SMP Thu Jun 14 13:51:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	ant
Personality	/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
git revision	master / 6fbcda6
ant	version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018
Default Java	1.8.0_172
Test Results	https://builds.apache.org/job/PreCommit-LUCENE-Build/89/testReport/
modules	C: lucene/suggest U: lucene/suggest
Console output	https://builds.apache.org/job/PreCommit-LUCENE-Build/89/console
Powered by	Apache Yetus 0.7.0 http://yetus.apache.org

This message was automatically generated.

[Legacy Jira: Lucene/Solr QA on Sep 07 2018]

mikemccand commented 6 years ago

This is a good idea, but we need to settle LUCENE-8343 first.

[Legacy Jira: Michael McCandless (@mikemccand) on Sep 10 2018]

mikemccand commented 6 years ago

I agree,

let's finalise LUCENE-8343 first and then I will take a look again to this contribution and update it !

[Legacy Jira: Alessandro Benedetti (@alessandrobenedetti) on Sep 10 2018]

mikemccand commented 5 years ago

❌ -1 overall

Vote	Subsystem	Runtime	Comment
-1	patch	0m 6s	LUCENE-8347 does not apply to master. Rebase required? Wrong Branch? See https://wiki.apache.org/lucene-java/HowToContribute#Contributing_your_work for help.

Subsystem	Report/Notes
JIRA Issue	LUCENE-8347
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12926717/LUCENE-8347.patch
Console output	https://builds.apache.org/job/PreCommit-LUCENE-Build/191/console
Powered by	Apache Yetus 0.7.0 http://yetus.apache.org

This message was automatically generated.

[Legacy Jira: Lucene/Solr QA on Jun 14 2019]

mikemccand commented 4 years ago

@mikemccand, @dsmiley I will take this back under my radar, fix the conflicts and take a look. LUCENE-8343 has been settled so we should be able to work on this now!

[Legacy Jira: Alessandro Benedetti (@alessandrobenedetti) on Jan 23 2020]

mikemccand commented 4 years ago

❌ -1 overall

Vote	Subsystem	Runtime	Comment
-1	patch	0m 7s	LUCENE-8347 does not apply to master. Rebase required? Wrong Branch? See https://wiki.apache.org/lucene-java/HowToContribute#Contributing_your_work for help.

Subsystem	Report/Notes
JIRA Issue	LUCENE-8347
JIRA Patch URL	https://issues.apache.org/jira/secure/attachment/12926717/LUCENE-8347.patch
Console output	https://builds.apache.org/job/PreCommit-LUCENE-Build/250/console
Powered by	Apache Yetus 0.7.0 http://yetus.apache.org

This message was automatically generated.

[Legacy Jira: Lucene/Solr QA on Jan 23 2020]

mikemccand / stargazers-migration-test

BlendedInfixSuggester to handle multi term matches better [LUCENE-8347] #347

Legacy Jira details